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FOREWORD 

Most students participate in academic and non-academic activities at school, and develop a sense of 
belonging — their friends are there, they have good relations with teachers and other students, and they 
identify with and value schooling outcomes. But many students are not engaged. They do not believe their 
school experience has much bearing on their future, and they do not feel accepted by their classmates or 
teachers. Gradually these students withdraw from school life, and become disaffected from school. Some 
disaffected students are disruptive in class, and exert a negative influence on other students. 

Can we meet the needs of students who have become disaffected from school? Can we identify schools 
that have high levels of student engagement, and if so, what factors contribute to their success? What is 
the relationship between student engagement and academic performance? These questions are of great 
concern to educators around the world. 

The OECD Programme for International Student Assessment (PISA) offers a unique opportunity to study 
student engagement across several countries as students approach the end of compulsory schooling. The 
data collected in PISA include information on students’ attitudes and values, as well as reliable and valid 
data on students’ literacy skills. The data also include information on students’ family backgrounds and 
on several features of the schools they were attending. This report examines several questions concerning 
students’ participation and sense of belonging. These two aspects of student engagement are considered 
important, not only because of their relationship with student learning, hut also because they represent a 
disposition towards schooling and life-long learning. 

The results indicate that there is considerable variation among countries in their levels of student engage- 
ment and in the prevalence of disaffected students. Moreover, the prevalence of disaffected students varies 
considerably within and among schools within most countries, and this variation is not attributable solely 
to students’ family background. The results also provide evidence that literacy performance and student 
engagement do not necessarily go hand-in-hand; in most countries there is a significant number of students 
with a strong literacy performance who are nevertheless disaffected from school. The analyses also iden- 
tify some of the school factors related to engagement, and provide evidence that achieving strong student 
engagement at school does not have to be at the expense of their reading performance. 

PISA is a collaborative effort, bringing together scientific expertise from the participating countries, 
steered jointly by their governments on the basis of shared, policy-driven interests. Participating countries 
take responsibility for the project at the policy level through a Board of Participating Countries. Experts 
from participating countries serve on working groups that are charged with linking the PISA policy objec- 
tives with the best available substantive and technical expertise in the field of international comparative 
assessment of educational outcomes. Through participating in these expert groups, countries ensure that 
the PISA assessment instruments are internationally valid and take into account the cultural and curricular 
contexts of OECD Member countries, that they provide a realistic basis for measurement, and that they 
place an emphasis on authenticity and educational validity. The frameworks and assessment instruments 
for PISA 2000 are the product of a multi-year development process and were adopted by OECD Member 
countries in December 1999. 

This report is the product of a concerted effort between the author, Jon Douglas Willms, the countries 
participating in PISA, the experts and institutions working within the framework of the PISA Consortium, 
and the OECD. The report was prepared by the OECD Directorate for Education under the direction 
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of Kooghyang Ro and Andreas Schleicher. Its development was steered by the Board of Participating 
Countries, chaired by Eugene Owen of the National Center for Education Statistics in the United States. 
Annex C of the report lists the members of the various PISA bodies as well as the individual experts 
and consultants who contributed to it and to PISA in general. Special thanks are extended to Elizabeth 
Fairbairn and Cara Fedick for their assistance in preparing the manuscript. 

The report is published under the responsibility of the Secretary- General of the OECD. 
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I CHAPTER 1 Student Engagement at School: A Sense of Belonging and Participation 

A definition of student engagement 

School is central to the daily life of many youths. They view schooling as essential to their long-term well- 
being, and this attitude is reflected in their participation in academic and non-academic pursuits. These 
students tend to have good relations with school staff and with other students — they feel that they belong 
at school. However, some youths do not share this sense of belonging, and do not believe that academic 
success will have a strong bearing on their future. These feelings and attitudes may result in their becoming 
disaffected from school (Finn, 1989; Jenkins, 1995). They may gradually withdraw from school activities, 
and in some cases participate in disruptive behaviour and display negative attitudes towards teachers and 
other students. Meeting the needs of youths who have become disaffected from school is perhaps the big- 
gest challenge facing teachers and school administrators. 

Researchers have recently used the term engagement to refer to the extent to which students identify 
with and value schooling outcomes, and participate in academic and non-academic school activities. Its 
definition usually comprises a psychological component pertaining to students’ sense of belonging at school 
and acceptance of school values, and a behavioural component pertaining to participation in school activities 
(Finn, 1989, 1993; Finn and Rock, 1997; Goodenow, 1993; Goo denow and Grady, 1993;Voelkl, 1995, 
1996, 1997; Wehlage et al. , 1989). The psychological component emphasises students’ sense of belonging 
or attachment to school, which has to do with feelings of being accepted and valued by their peers, and by 
others at their school. Another aspect of the psychological component concerns whether or not students 
value school success — do they believe that education will benefit them personally and economically 
(Johnson et al., 2001). Students who do not feel they belong at school, or reject school values, are 
often referred to in the literature as alienated or disaffected. The participation component of engagement 
is characterised by factors such as school and class attendance, being prepared for class, completing 
homework, attending lessons, and being involved in extra-curricular sports or hobby clubs. 

In this report, the term student engagement is used in this broad sense to refer to students’ attitudes towards 
schooling and their participation in school activities. The term disengaged from school is used to characterise 
students who do not feel they belong at school and have withdrawn from school activities in a significant 
way. Student engagement, as used in this report and in the literature, differs from reading engagement, 
as used in the OECD thematic report, Reading for Change: Performance and Engagement across Countries 
(OECD, 2002). Reading engagement refers specifically to students’ motivation and interest in reading, 
and the time students spend reading for pleasure and reading diverse materials. 

Most of the research on engagement has been concerned with its relationship to academic achievement 
and whether or not students are likely to complete secondary school. Variables describing engagement 
have therefore usually been treated in analyses as predictors of other schooling outcomes, particularly 
academic performance. This report considers sense of belonging and participation as important schooling 
outcomes in their own right. Engagement is seen as a disposition towards learning, working with others 
and functioning in a social institution, which is expressed in students’ feelings that they belong at school, 
and in their participation in school activities. It has yet to be examined whether disengagement from 
school during the adolescent years will have longer term effects. However, it can certainly be expected 
that students’ attitudes towards school and their participation strongly affect their decision whether or 
not to pursue post-secondary studies. It is known that youths who have behavioural problems tend to be 
disaffected from school (Offord and Waters, 1983): longitudinal studies that have followed young people 
with behavioural problems into adulthood have found that nearly one-half of them continue to suffer from 
psychological and social difficulties as adults (Offord and Bennett, 1994). Thus, engagement is probably 
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closely tied to students’ economic success and long-term health and wellbeing, and as such deserves to be 
treated alongside academic achievement as an important schooling outcome. Moreover, engagement is not 
an unalterable trait of individuals, stemming solely from students’ genetic make-up or their experiences at 
home. Rather, it entails attitudes and behaviours that can be affected by teachers and parents, and shaped 
by school policy and practice. 

The Programme for International Student Assessment (PISA), conducted by the Organisation for 
Economic Co-operation and Development (OECD), offers an opportunity to study student engagement 
across several countries. PISA is an assessment of how well young adults, at age 15, are able to use the 
knowledge and skills they have acquired to meet the challenges facing them as they approach completion 
of their compulsory schooling. Thirty-two countries participated in the first PISA survey in 2000. 
It included 28 Member countries of the OECD, and four non-OECD countries. In 2002, another 
1 1 1 non-OECD countries completed the survey. The PISA assessment includes a series of written tasks 
that measure reading, mathematical and scientific literacy, and a questionnaire regarding students’ family 
background, experiences at school and attitudes towards learning. The focus of PISA in 2000 and 2002 
was on reading literacy, and therefore included an extensive set of tasks in this domain. In 2003, the 
emphasis will be on mathematics literacy, and in 2006, on scientific literacy. The first international report, 
Knowledge and Skills for Life: First Results from the PISA 2000 (OECD, 2001c!), focuses on student performance 
in reading, mathematics and scientific literacy, and how performance in these domains relates to students’ 
family background and features of the schools they attend. 

This report examines students’ sense of belonging and participation at school, two of the most important 
measures of student engagement. Criteria are established to identify students who could be considered 
disaffected — those who have a very low sense of belonging at school compared with the majority of their 
peers — and those who are consistently absent. The analyses provide estimates of the prevalence of students 
who have a low sense of belonging or low participation for each of the 42 countries that participated in 
PISA 2000, and for the schools within each country. The analyses also assess the strength of the relationship 
between these two aspects of engagement with performance in reading, mathematics and scientific literacy. 
Finally, the analyses examine the relationships between these two measures of engagement with gender, 
family structure and socio-economic status, and the characteristics of the schools students were attending 
when surveyed. The results have important implications for teachers, school administrators and policy- 
makers, as they stress the importance of viewing engagement as a fundamental schooling outcome. 

Student engagement and academic success 

Most recent studies of student engagement treat it as a predictor of academic achievement, inferring that 
being disengaged, or disaffected from school, causes poor academic achievement. However, the theoretical 
literature argues that it is low achievement that causes students to withdraw from school, or that engagement 
and academic achievement go hand-in-hand. Bloom (1976) noted: 

“At the other extreme are the bottom third of students who have been given consistent evidence 
of their inadequacy. . .over a period of five to ten years. Such students rarely secure any positive 
reinforcement in the classroom. . . from teachers or parents. We would expect such students to be 
infected with emotional difficulties [and to] exhibit symptoms of acute distress and alienation from 
the world of school and adults.” 
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I CHAPTER 1 Student Engagement at School: A Sense of Belonging and Participation 

Longitudinal research on child development suggests that there is a core set of risk factors, including 
poverty, poor temperament, cognitive problems, learning disabilities and physical and mental handicaps, 
that are evident in many children when they enter school. Children who display behaviour problems or 
cognitive deficits during the early years of schooling are vulnerable, in the sense that without concerted 
and prolonged intervention their chances of succeeding at school or leading healthy and productive 
lives are diminished (Willms, 2002a). Many of these children struggle with learning to read (Rowe and 
Rowe, 1992), and their problems worsen when mathematics and other school subjects place greater 
demands on reading skills. By the middle school grades ( e.g ., grades 5 to 7), many of these children display 
a low commitment to educational activities, a disaffection towards school, poor social bonding, and poor 
peer relations. These characteristics continue into the secondary school years, which place these children 
at a very high risk of conduct disorders, low achievement, and early school withdrawal (Coie and Jacobs, 
1993; Coie et al., 1993; Hawkins et al., 1988; Rumberger, 1983, 1995;Yoshikawa, 1994). Moreover, the 
longer-term health and wellbeing of these children is also compromised (Power et al., 1991; Pulkkinen 
and Tremblay, 1992; Rodgers, 1990). To summarise, the literature suggests that there are risk factors for 
both school disaffection and poor achievement that are evident when children enter school, and these risk 
factors are cumulative and predictive of longer-term life outcomes. 

However, it cannot be inferred that low student engagement during the secondary school years is simply 
the consequence of family-related risk factors, such as poverty, low parental education or poor cognitive 
ability. Many children who grow up in poor families, or have behaviour problems and cognitive difficulties 
during their primary school years, prove to be remarkably resilient. They graduate from high school and 
lead happy and productive lives (Werner and Smith, 1992). Conversely, the onset of antisocial behaviours 
that are associated with engagement (Jenkins, 1995) begins late for many adolescents (Moffit et al. , 1996). 
Small classroom- and school-based studies have found that the correlation between academic achievement 
and engagement is moderate — generally between 0.25 and 0.30 (Goodenow, 1993; Voelkl, 1 995) — which 
suggests that there are many students with high achievement who are not engaged and vice versa. Moreo- 
ver, there is ample evidence that the school environment has a strong effect on children’s participation and 
sense of belonging (Bryk andThum, 1989; Fine, 1986; Finn and Voelkl, 1993; Johnson et al., 2001; Lee 
and Smith, 1993). 

Engagement and the school environment 

A few recent studies have treated engagement as an outcome variable, and have attempted to explain why 
schools vary in their levels of engagement (Finn and Voelkl, 1993; Johnson et al., 2001; Lee and Smith, 
1993). Lee and Smith (1993) found modest effects of school restructuring — heterogeneous grouping, 
team teaching and reducing the extent to which schools are organised by subject-area departments — on 
levels of student engagement. Finn and Voelkl (1993) found that attendance was less consistent and at-risk 
students were less engaged in schools where rules were not enforced rigidly. This is consistent with Rum- 
berger’s (1995) finding that students are less likely to drop out of school before graduation when students 
perceive the discipline policy is fair. 

Johnson et al. (2001) also examined the compositional or contextual effects associated with school mean 
grade-point average and the percentage of own race / ethnicity students in the school. The term contextual 
effects is used in the literature to refer to the effects of a classroom or school’s aggregate characteristics, 
such as the average level of socio-economic status or average ability, on student outcomes, over and above 
the effects associated with students’ individual characteristics (Willms, 2001). Johnson et al. (2001) found 
small but significant effects of racial and ethnic composition on levels of attachment (sense of belonging), 
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but no significant effects for participation (skipping school, paying attention in class, doing homework). 
However, they did find that levels of participation were strongly associated with the mean grade-point 
average of the school. Significant contextual effects were also reported by Rumberger (1995): students 
were more likely to drop out of school if they attended schools with a high percentage of minority students 
or a low average socio-economic status. 

An important question addressed in this study is, therefore, whether the average socio-economic status of 
the school has an effect on students’ engagement, over and above the effects associated with students’ own 
family backgrounds. Contextual effects have been found to have strong effects on student achievement in 
a number of educational studies (Gamoran, 1992; Willms, 1999a), including the first international report 
for PISA (OECD, 2001a). They are important for educational policy, as they suggest that when students 
are segregated along social-class lines into different schools or educational programmes, students from 
disadvantaged backgrounds tend to have markedly worse outcomes. 

The Programme for International Student Assessment (PISA) as a study of student 
engagement 

The OECD PISA has a number of strengths for examining student engagement. First, it is based on large, 
nationally representative samples of 15-year-old students in 45 countries. This allows comparison of 
results across countries. Second, PISA has effective measures of literacy skills in reading, mathematics and 
science at the student level. It also includes several items in the student questionnaire that directly pertain 
to student engagement.These items are used in this report to construct outcome measures describing stu- 
dents’ sense of belonging and participation. The study also has a comprehensive set of measures describing 
several aspects of students’ family background. Therefore, it is possible to examine for each country the 
relationships among achievement and engagement outcomes and their relationships with a comprehen- 
sive set of variables describing students’ family background. Third, the sample design entailed sampling 
schools within countries and students within schools. With this type of multilevel design it is possible to 
discern whether disaffected students tend to be concentrated in a relatively small number of schools, or 
fairly evenly distributed across schools within each country. Fourth, PISA includes extensive information 
describing school context, school resources, school policy and practice, and classroom practice. Multilevel 
statistical models (Bryk and Raudenbush, 2002) can be used to explore the relationship between levels of 
student engagement and family and school factors (See Box 1.1). 

The remainder of this report comprises four chapters. The next chapter describes the two engagement 
measures used in this study: sense of belonging and participation. Criteria are established to identify youths 
with low scores on these measures, making it possible to estimate the prevalence of students who could be 
considered disaffected. The analyses examine the extent to which the average scores for sense of belonging 
and participation, and the prevalence of youths with very low scores on these two measures vary among 
countries. An important policy issue is whether efforts to reduce student disaffection should he targeted 
at particular schools, or whether it requires a more universal intervention that is focused on all schools. 
The analyses address this issue by estimating the prevalence of disaffected students within each school, and 
determining the range of prevalence estimates across schools within each country. 
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Box 1.1 Hierarchical Linear Modelling (HLM) 

Survey data in the social sciences are often structured hierarchically. In the case of PISA, for example, 
students are nested within schools, which are nested within countries. Hierarchical linear mod- 
eling (HLM) is a specialised regression technique designed to analyse hierarchically structured data 
(Goldstein, 1995; Bryk and Raudenbush, 2002). With traditional regression approaches, such as 
multiple regression and logistic regression, an underlying assumption is that the observations are 
independent. This means that the observations of any one individual are not in any way systemati- 
cally related to the observations of any other individual. The assumption is violated, however, when 
some of the children sampled are from the same family, or the same classroom or school. When the 
assumption of independence is violated, the regression coefficients can be biased, and the estimates 
of standard errors are smaller than they should be. Consequently, there is a risk of inferring that a 
relationship is statistically significant when it may have occurred by chance alone. 

In addition, the interest from a policy perspective is usually in the relationships within schools, 
whether these relationships vary among schools, and if so, whether the variation is related to school 
characteristics. For example, the average level of students’ engagement (either sense of belonging 
or participation), and the relationship between engagement and socio-economic status, may vary 
among schools within a given country. The policy analyst may be interested in whether schools with 
high average engagement and more equitable engagement have smaller class sizes, different kinds 
of instructional techniques, or differing forms of school organisation (Lee et al . , 1990; Raudenbush 
andWillms, 1995). 

The basic idea underlying HLM is that there are separate analyses for each school (or the emit at the 
lowest level of a hierarchical structure), and the results of these analyses — usually regression coef- 
ficients — become the dependent variables for analyses at the school level (or at the next level of the 
hierarchy). Willms (1999 b) provides an introduction to HLM for the non- statistical reader, with a 
general discussion of its applications to educational policy issues. Goldstein (1995), and Bryk and 
Raudenbush (2002) provide comprehensive texts on HLM that can be understood fairly easily by 
those familiar with basic regression analyses. 


Chapter 3 of the report examines the relationships between the three measures of literacy performance 
— reading, mathematical and scientific literacy — and the two engagement outcomes: sense of belonging 
and participation. The simplest and most straightforward approach is to estimate the correlations among 
these five variables. However, there is a second-order question that asks whether these variables are 
strongly related at the school level; for example: do schools with high literacy performance also tend to 
have strong student engagement, and vice versa? It is likely to be the case, as the same school processes 
that affect literacy performance, such as a positive climate for learning and maintaining discipline, positive 
student-teacher relations and parental involvement, probably also engender a sense of belonging at school, 
and increased participation. However, it may be that high expectations for literacy performance and an 
emphasis on academic subjects may cause many students to become disaffected from school. The analyses 
employ a multivariate, multilevel statistical model to estimate the correlations among these variables at 
both the student and school levels. The results of these correlational analyses call for a different kind of 
multivariate analysis, which asks whether students cluster into particular groups based on their profile of 
scores on the engagement and literacy measures. Therefore, cluster analyses are conducted for each coun- 
try, revealing patterns of association that have important implications for educational policy. 
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Chapter 4 extends the analyses that treat sense of belonging and participation as outcomes in their own 
right. The first set of analyses examines the relationship between student disaffection — that is, low sense of 
belonging and low participation — and the students’ gender, family structure, and whether they are living 
in a high or low socio-economic status family. These relationships differ significantly among countries, 
and therefore separate results are displayed for each country. The second set of analyses examines the 
effects associated with school factors. These relationships also differ among countries; however, there is 
not a large enough sample size within each country to examine these relationships separately by country. 
The analyses therefore estimate the average relationship for all participating countries combined. These 
analyses reveal that four school factors have particularly strong relationships with the two engagement 
outcomes. The section concludes with a presentation of how these factors vary among countries. 

The final chapter, Chapter 5, presents a summary of the study, discusses its implications for educational 
policy, and makes recommendations for further research. 


Notes 


1 . The data results for Romania are not included in this report due to delayed submission of data. 
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Data underlying the figures 

The data referred to in Chapters 2 to 4 of this report are presented in Annex B and with additional 
detail, on the web site www.pisa.oecd.org. 

Calculation of international averages 

There are three kinds of international averages: 

The OECD average, sometimes also referred to as the country average, is the mean of the data 
values for all OECD countries for which data are available or can be estimated. The OECD average 
can be used to see how a country compares on a given indicator with a typical OECD country. The 
OECD average does not take into account the absolute size of the student population in each coun- 
try, i.e . , each country contributes equally to the average. 

The OECD total, sometimes also referred to as the total average, takes the OECD countries as 
a single entity, to which each country contributes in proportion to the number of 1 5 -year-olds 
enrolled in its schools. It illustrates how a country compares with the OECD area as a whole. 

Readers should, therefore, keep in mind that the terms OECD average and OECD total refer to the 
OECD countries included in the respective comparisons. 

The Netherlands are excluded from the estimation of these three averages because low response 
rates preclude reliable estimates of mean scores. In the case of other countries, data may not be 
available for specific indicators, or specific categories may not apply. 

Index of central tendency 

In order to give an overview of the average trend observed among countries, the average of the 
OECD countries is reported. In some cases the OECD average is not reported because an average 
of within- country relationships does not provide meaningful inf ormation about relationships across 
all countries. 

Reporting of student data 

The report usually uses 15 -year-olds as shorthand for the PISA target population. In practice, this 
refers to students w r ho were aged between 15 years and 3 (complete) months and 16 years and 
2 (complete) months at the beginning of the assessment period and who were enrolled in an edu- 
cational institution, regardless of the grade level or type of institution and of whether they are full- 
time or part-time students. 

Reporting of school data 

The principals of the schools in which students were assessed provided inf ormation on their schools 
characteristics by completing a school questionnaire. Where responses from school principals are 
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presented in this publication, they are weighted so that they are proportionate to the number of 
15 -year-olds enrolled in the school. 

Rounding of figures 

Because of rounding, some figures in tables may not exactly add up to the totals. Totals, differences and 
averages are always calculated on the basis of exact numbers and are rounded only after calculation. 

Further documentation 

For further information on the PISA assessment instruments and the methods used in PISA, see the 
Knowledge and Skills for Life: First Results from PISA 2000 (OECD, 2001), PISA 2000 Technical Report 
(OECD, 2002u) and the PISA Web site ( www.pisa.oecd.org ). 
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I CHAPTER 2 Student Engagement at School: A Sense of Belonging and Participation 

Introduction 

The construct of engagement generally includes an affective or feeling component pertaining to students’ 
sense of belonging at school and how much students identify with and value schooling outcomes, and a behav- 
ioural component pertaining to students’ participation in academic and non-academic activities. This chapter 
begins with a description of how these two dimensions of engagement were measured in PISA and how they 
were used to construct two indicators of student disaffection. It then examines the extent to which countries 
vary in their levels of engagement and in the prevalence of disaffected youths. The analyses then provide esti- 
mates of the extent to which the prevalence of disaffected students varies among schools within each country. 
The chapter concludes with a discussion of the policy issues relevant to the findings in this chapter. 

How student engagement is measured in PISA 

The construct of engagement derived from the PISA results for this report has two dimensions: sense of 
belonging and school attendance. Details of the student engagement measures and the construction of 
PISA engagement index are described in Annex A. 

Sense of belonging was based on students’ responses to six items describing their personal feelings about 
being accepted by their peers and whether or not they felt lonely, “like an outsider” or “out of place”. Like 
literacy performance or virtually any schooling outcome, sense of belonging is affected by students’ expe- 
riences at home and in their community, as well as by their school experience. 

The second component is participation, which is measured by the frequency of absence, class-skipping and 
late arrival at school during the two weeks prior to the PISA 2000 survey. There are two issues concerning 
the validity of participation measure that warrant discussion. One issue is that the measure of participation 
could be more extensive. It was measured in this study with a rather narrow focus on student absenteeism. 
Part of the problem is that the very nature of school participation varies considerably among countries, 
making it difficult to measure participation with a broader focus that includes time spent on homework, 
participation in classroom discussions and involvement in sports and other extra-curricular activities. Also, 
the meaning of the construct itself undoubtedly has a cultural component and thus varies among countries 
and among subgroups within countries. Although a broader measure of participation would be desirable, 
student absenteeism is the most important aspect of participation. This is because there is generally a pro- 
gression in students’ withdrawal from school. For example, students who are mildly disaffected may spend 
less time on homework and participate less in classroom discussions; then they may skip a few classes from 
time to time; eventually, they may skip classes and miss full days on a regular basis (see Finn, 1989). 

The second issue pertains to how participation is measured. A number of students may have missed school 
because of illness or for other legitimate reasons. Thus, the measure of participation should be viewed as a 
measure of absenteeism, rather than as a measure of truancy, or participation per se. Related to this issue is 
the problem that students who are absent more often are less likely to have been at school on the day the 
PISA questionnaires were administered. This would result in a bias such that the prevalence of low par- 
ticipation is under-estimated. The PISA data include a design weight that adjusts for non-response at the 
school level, and to some extent this would reduce this bias. However, when considering the magnitude of 
the prevalence of low participation, it is likely that it is to some extent under-estimated. 

The two indicators of student engagement at school were also constructed from these measures. Students 
were considered to have a low sense of belonging if they scored below 3.0 on the sense of belonging scale 
(before standardisation). These students, on average for the six items, responded “disagree” or “strongly 
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disagree” more frequently than “agree” or “strongly agree”. Students who feel that they “belong” can be 
expected on average at least to “agree” with the positive statements and “disagree” with the negative ones. 
Those with a lower average score are classified as having a “low sense of belonging” . This does not mean 
that they express negative attitudes overall, but they do in at least one respect. Also, analyses of the dis- 
tribution of the scaled scores suggested that TO was an appropriate cut-point. The sense of belonging 
scale was negatively skewed — 0.70 for participating OECD countries — which indicates that there were 
a number of students with exceedingly low scores. Almost one-quarter (24.5 per cent) of all students 
scored below 3.0 on the unstandardised scale, which corresponded to scores at or below 426 or lower on 
the standardised scale. There is a marked break in the distribution at this point. Students with scores of 
3.0 or higher had scaled scores of 460 or higher. Thus, the criterion used for classifying students as having 
a low sense of belonging has a simple substantive interpretation and is based on a significant break in the 
observed distribution of scores. 

There was another reasonable cut-point for this scale, which would consider youths to have a “low sense 
of belonging” if they scored below 2.8 on the sense of belonging scale before standardising. Defining low 
sense of belonging in this way yielded a prevalence of 16.4 per cent rather than 24.5 per cent. However, 
comparisons across countries in the prevalence of students with a low sense of belonging are not substan- 
tially affected by this choice: the correlation at the country level between a variable based on this cut-off 
score and the one selected was 0.97. 

Students were considered to have low participation if they scored less than or equal to 10 on the unstand- 
ardised participation scale. This also has an appealing substantive interpretation. For example, all students 
were considered to have low participation if they responded “1 or 2 times” to all three items, or “3 or 
4 times” to “miss school”, or “3 or 4 times” to both “skip classes” and “arrive late for school”. The participa- 
tion variable was also strongly skewed negatively — 1.82 for participating OECD countries. As with the 
sense of belonging scores, this indicates that there are a number of students with exceedingly low scores. 
With these criteria set at 10 or lower on the participation scale, 20 per cent of students in participating 
OECD countries were classified as having low participation. 

As with sense of belonging, there are other reasonable cut-points to define low participation. Using a 
cut-point of 9.0 instead of 10.0 yielded a prevalence of 13.8 per cent, while a cut-point of 1 1.0 yields a 
prevalence of 29 per cent. But in this case also, the choice of cut-points has little effect on international 
comparisons: the correlation at country level between the variable with the lower cut-off score and the 
one selected was 0.99, while the correlation between the variable with the higher cut-off score and the 
one selected was 0.96. 

Although the choice of cut-points does not materially affect international comparisons, they do of course 
affect the estimates of prevalence. Thus, when making substantive interpretations of low sense of belonging 
or low participation, the reader needs to be cognisant of the definitions set out above. 

Variation among countries in student engagement 

Figure 2.1 displays the country mean for the sense of belonging and participation indices. Descriptive 
statistics for the two measures of engagement for each country are also reported in Tables 2.1 and 2.2. 
The first column of each table is the mean score on the measure, which is followed in the second column 
with the standard error of the mean. The OECD average was fixed at 500, and therefore countries with 
scores above 500 have more favourable engagement scores, while those with scores below 500 have less 
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favourable scores. The standard error gives an indication of how accurately the outcome was measured for 
each country, which depends mainly on the sample size. The 95 per cent confidence interval encompasses 
the mean score, below and above two standard errors. If PISA could be repeated several times, the mean 
score would be expected to fall within that interval 19 times out of 20. The third column gives the stand- 
ard deviation of the outcome measure. This is an indication of the overall variability of scores within each 
country. The standard deviation of each measure was set to 100 for all OECD countries. 

OECD countries varied in their levels of sense of belonging, ranging from 461 for Korea and Poland to 527 
for Sweden (see Table 2.1). However, most countries had scores that did not differ significantly from the 
OECD average. Among the OECD countries, only three countries had scores that were significantly above 
515: Sweden, Austria, and Switzerland. 1 Five countries had scores below the OECD average: Belgium, 
the Czech Republic, Japan, Korea and Poland. Among the non-OECD countries, two countries, Brazil 
and Israel, had scores that were statistically significantly above the OECD average, while eight of the other 
non-OECD countries had relatively low scores, at least 15 points below the OECD average. 


Figure 2.1 
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Source: OECD PISA Database, 2003. Tables 2.1 and 2.2. 
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Levels of participation vary considerably more among countries. Among the OECD countries, three coun- 
tries had scores significantly above the OECD average: Japan, Korea and Germany. In contrast, six countries 
had average scores below the OECD average: Canada, Denmark, Greece, New Zealand, Poland and Spain. It 
is interesting to note that Denmark and Sweden had relatively high scores on the sense of belonging measure, 
but relatively low scores on the participation measure. Among the non-OECD countries, three countries had 
scores that were significantly above the OECD average: Hong Kong-China, Indonesia and Liechtenstein. All 
of the other non-OECD countries, except Albania, Latvia, FYR Macedonia and Thailand, had relatively low 
scores, significantly below the OECD average. The correlation at the country level between sense of belong- 
ing and participation among non-OECD countries is also negative: -0.44. 

Variation among countries in low sense of belonging and low participation 

Figure 2 . 2 displays the prevalence of youths considered to have a low sense of belonging and participation 
for the 28 OECD countries and the 14 non-OECD countries. Standard errors of these frequencies are 
displayed in Table 2.3. 

The prevalence of students with a low sense of belonging in most countries did not differ substantially 
from the OECD average of 24.5 per cent. There were four countries with averages above 30 per cent: Bel- 
gium (32 per cent), Japan (38 per cent), Korea (41 per cent) and Poland (41 per cent). The prevalence was 
below 20 per cent in only three countries: Hungary (19 per cent), Sweden (18 per cent) and the United 
Kingdom (17 per cent). 

The prevalence of students with a low sense of belonging in the non-OECD countries varied consider- 
ably, from 17 to 40 per cent. "Hie prevalence was above 30 per cent in 6 of the 14 countries: Albania 
(40 per cent), Hong Kong-China (33 per cent), Latvia (36 per cent), Peru (37 per cent), the Russian Fed- 
eration (33 per cent) and Thailand (33 per cent). The prevalence was below 20 per cent in Brazil (17 per 
cent). Figure 2.2 displays the prevalence of students considered to have low participation. 

The prevalence of students with low participation varies more among countries than the prevalence of 
students with a low sense of belonging. While the average level of low participation students among 
OECD countries is 20.0 per cent, the prevalence was above 25 per cent in six of the 28 countries: 
Canada (26 per cent), Demnark (33 per cent), Greece (29 per cent), New Zealand (27 per cent), Poland 
(29 per cent) and Spain (34 per cent). In contrast, the prevalence was below 1 5 per cent in four countries: 
Germany (13 per cent), Japan (4 per cent), Korea (8 per cent) and Luxembourg (13 per cent). 

Low participation appears to be a greater problem in the non- OECD countries. The prevalence of low 
participation was above 25 per cent in seven of the fourteen countries. It was below 15 per cent in Hong 
Kong-China (3 per cent) and Liechtenstein (9 per cent). 

Variation among schools in low sense of belonging and low participation 

The prevalence of students with a low sense of belonging may also vary considerably among schools within 
each country. Determining the extent of this variation is important for at least two reasons. If there is 
considerable variation among schools, then it may be more efficient to target certain schools for interven- 
tion, whereas if the prevalence is fairly uniform across most schools in a country, then a more universal 
intervention is likely to be preferable. The second reason is that if there is considerable variation among 
schools in the prevalence of disaffected students, it may be possible to discern whether particular school 
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Prevalence of students with low sense of belonging and low participation 
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factors are related to either sense of belonging or participation, thereby providing some direction for what 
kinds of interventions might be most effective. 
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For each country, therefore, the prevalence of students with a low sense of belonging and low participation 
was calculated for each school. These were calculated for low sense of belonging and for low participation 
by fitting separately for each country a null multilevel logistic regression model on the dichotomous ( i.e 
two-category) outcome measure. The multi-level analysis yields estimates of the prevalence of low sense of 
belonging or low participation for each school. These estimates are adjusted to take account of the sample 
size with which each estimate was calculated. With the sample sizes in PISA, it is not possible to get a very 
accurate estimate of the prevalence for any particular school; however, the analysis gives an accurate por- 
trayal of the variation in prevalence estimates across all schools in a given country. 


Figure 2.3 


Variation among schools in the prevalence of low sense of belonging, by country 
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1 . Response rate is too low to ensure comparability. 

Source: OECD PISA database, 2003. For data see www.pisa.oecd.org. 
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Figures 2.3 and 2.4 display the variation in the estimates of the 
prevalence of students with low sense of belonging and low par- 
ticipation within each country. These are displayed as percentile plots 
(see Box 2.1), which show the median prevalence for all schools in 
die country, and the 5 th , 2 5 th , 75 th and 95 th percentiles for the distri- 
bution of prevalence estimates for all schools in the country. 

On average, across 28 OECD countries, the median prevalence 
was 24 per cent. The average prevalence at the student-level for all 
OECD countries was 25.5 per cent. Within every country, except 
Iceland, New Zealand and Sweden, the prevalence of students 
with a low sense of belonging varied significantly among schools. 
The average inter-quartile range was 4.8 per cent, and the average 
range from the 5 th to the 95 th percentiles was 12.6 per cent. In three 
countries, Korea, Luxembourg and Poland, the range exceeded 
20 per cent, indicating considerable variation among schools. 


Figure 2.4 


Variation among schools in the prevalence of low-participation students, by country 
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On average, for the non-OECD countries, the median prevalence of students with a low sense of belong- 
ing was 27.7 per cent. In all countries except Brazil and Liechtenstein, the prevalence varied significantly 
among schools within each country. On average, the inter-quartile range was 6.9 per cent, and the average 
range (from the 5 th to the 95 th percentile) was 19.9 per cent. 

On average, across the 28 OECD countries, the median prevalence of low participation students was 
19 per cent, which is slightly lower than the average prevalence at the student-level of 20 per cent. The 
prevalence of low participation students varied significantly among schools in every OECD country. The 
average inter-quartile range was 7.3 per cent, and the average range (from the 5 th to the 95 th percentiles) 
was 19.8 per cent. Note that these figures indicate that there was considerably more variation among 
schools in the prevalence of students with low participation than for low sense of belonging. The range 
exceeded 25 per cent in six countries: Hungary, Italy, Poland, Spain, Switzerland and the United States. 

The average median for the non-OECD countries was 24.9 per cent. In all countries, except Liechtenstein, 
the prevalence of low participation students varied significantly among schools. The average inter-quartile 
range was 8.3 per cent, and the average range (from the 5 th to the 95 th percentiles) was 22.0 per cent. 

Conclusion 

The analyses in this chapter examined the distribution of scores on two measures of student engagement : 
sense of belonging and participation. Students who had very low scores on these measures were considered 
to be disaffected from school, and the analyses also estimated the prevalence of disaffection based on these 
two indicators. The following findings are particularly noteworthy: 

(a) There is a high prevalence of students who can be considered disaffected from school 
in terms of their sense of belonging or their participation. On average, across the OECD 
countries, about one in four students are classified as having a low sense of belonging, 
and about one in five students has very low participation. The prevalence of both types 
of disaffection was higher among non -OECD countries. These results depend on the criteria 
(or cut-off scores) established in this study to determine whether a student was considered to have a 
low score on the sense of belonging and participation measures. Although it could be argued that the 
criteria were too liberal, thereby yielding high prevalence estimates, a case can easily be made that 
the criteria are quite conservative. Most students reported that they agreed, either moderately or 
strongly, with most statements regarding their sense of belonging. These students scored above 3.0 on 
the unstandardised scale. However, about one in every four students scored below 3.0. These students 
did not agree with all statements, and had indicated that they disagreed, either moderately or strongly, 
with at least one of the statements concerning their sense of belonging at school. Students were not 
considered to have low participation if they missed school 1 or 2 times in the previous two weeks of 
school, or had skipped classes and arrived late for school on 1 or 2 occasions during the previous two 
weeks. To be considered a student with low participation, a student had to have missed school at least 
3 or 4 times, or had a combination of missing school, skipping classes and arriving late for school that 
resulted in missing about 3 or more days of schooling in the previous two weeks. 

(b) Countries did not vary substantially in their levels of sense of belonging, or in the prev- 
alence of students with a low sense of belonging. Most countries had mean scores on the sense 
of belonging scale that were within 1 5 points of the OECD average of 500. Similarly, the prevalence of 
disaffected students across the OECD countries was close to the average prevalence of 24.5 per cent 
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in most countries. There was somewhat more variation in the prevalence of students with a low sense 
of belonging among non-OECD countries. 

(c) Countries varied considerably more in their levels of participation and in the preva- 
lence of low participation students. Mean scores on the participation scale were below 450 in 
two countries (Bulgaria and Israel), and above 550 in two countries (Japan and Hong Kong-China). 
Similarly the prevalence of low participation students varied substantially: it was less than 5 per cent 
in Hong Kong-China, while it was over 30 per cent in Bulgaria, Denmark, Israel and Spain. 

(d) In nearly every country, the prevalence of disaffected students varied significantly 
among schools. It is possible with the PISA measures to distinguish among schools that have a 
particularly high or low prevalence of disaffected students, even though the data are not sufficient for 
reporting results for individual schools. The results suggest that the variation among schools in the 
prevalence of students with a low sense of belonging is less than that of low participation. The analyses 
that follow in subsequent chapters examine to what extent some of this variation is attributable to 
family and school factors. 

These findings have important implications for educational policy. First, they indicate that disaffection from 
school is not limited to a small minority of students. Sense of belonging and participation are important 
schooling outcomes that deserve attention in nearly every country participating in PISA. While the preva- 
lence of students with a low sense of belonging or low participation varied significantly among schools in 
nearly every country, the analyses also found that all schools in nearly every country had a prevalence of 
low sense of belonging of at least 15 per cent, and a prevalence of low participation of at least 10 per cent. 
This suggests that virtually all schools need to deal with problems associated with disaffection, and thus 
most countries cannot adequately address the problem with interventions that are targeted at particular 
schools. 

The analyses also found that some countries had a relatively high prevalence of students with a low sense of 
belonging, but a low prevalence of students who were regularly absent. This suggests that policies directed 
at increasing student engagement need to consider students’ sense of belonging and their participation as 
separate issues. In the next chapter, the relationships among the engagement variables are considered in 
greater detail, alongside measures of literacy performance. 


Notes 


1 . Although the mean score for Germany is above 51 5, it is not significantly greater than 51 5 at a probability level of 0.05. This 
criterion is also applied in comparing the means scores for other countries with particular threshold values. 
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Introduction 

The first international report for PISA, Knowledge and Skills Jot Life (OECD, 2001a), provided convincing 
evidence that not only did countries differ in their average levels of literacy performance, but also that 
there was considerable variation in performance among schools within each country. The results in the 
previous chapter indicate that schools vary in the prevalence of students who are disaffected from school . 
A common approach to the study of engagement is to presume that it precedes academic outcomes, and 
that when students become disengaged from school, their academic performance begins to suffer. This may 
be the case for some students; however, an equally plausible model is that a failure to succeed in academic 
work at school results in student disaffection and the withdrawal from school activities. A third model, 
which is equally plausible, is that a range of other factors, including individual, family and school factors, 
jointly influence both engagement and academic outcomes. It may also be that the causal relationships 
differ, depending on the students’ temperament, academic ability, and family and school contexts. An 
understanding of the causal mechanisms associated with engagement and academic achievement is central 
to educational policy, in that it affects decisions about when and how to intervene. 

The PISA study cannot determine the causal relationships among engagement and achievement outcomes. 
However, it can provide an indication of how strong the relationships are among these outcomes at age 1 5, 
and determine how strongly they are related to family and school factors. This chapter examines the 
relationships among engagement and achievement outcomes using two different approaches. The first 
approach employs a multivariate, multilevel analysis to estimate the strength of the correlations among 
these variables at the individual and school levels. The second approach employs a statistical technique 
called cluster analysis. Its aim is to discern whether there are certain “clusters” or types of students 
characterised by their profile of engagement and achievement outcomes. 

The relationships between student engagement and literacy skills 

The two engagement outcomes (sense of belonging and participation) and the three measures of literacy 
skills (reading, mathematics and science) are analysed as outcome measures together in a single analysis. 
The analysis is multilevel because it takes account of the hierarchical structure of the data, that is, students 
are nested within schools in each country (see Box 1.1). The analysis discerns whether students who are 
more engaged in schooling tend to have better literacy skills and vice versa. The correlations among two 
outcome variables can be partitioned into within- and between-school components. The within-school 
component indicates how closely two variables are related among students within the same school. The 
school-level component indicates whether schools that have higher average scores on one outcome meas- 
ure also tend to have higher average scores on the other outcome measure, and vice versa. 

The multivariate, multilevel analysis was conducted separately for each of the 42 participating countries. 
The results include 10 student-level correlation coefficients and 10 school-level coefficients. These are 
presented in Tables 3 . 1 and 3.2. The results indicate that these relationships vary considerably among 
countries, but generally the relationships among the two engagement measures are much weaker than the 
relationships among the three measures of literacy performance, at both the individual and school levels. 
Also, the relationships between sense of belonging and the measures of literacy performance tend to be 
weak in most countries, while the relationships between participation and literacy performance are some- 
what stronger. 
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Figure 3.1 shows the average relationships among these variables for all participating OECD countries. 
Student-level correlations are shown below the diagonal, while school-level correlations are shown above 
the diagonal. At the student level, the average correlation between sense of belonging and participation 
is only 0.07. This is a very weak correlation, which suggests that they are markedly different outcome 
measures. There may be many students who lack a sense of belonging, but despite these feelings, they 
attend school regularly. Similarly, there may he many students who have a strong sense of belonging, hut 
miss school often, and regularly skip classes and arrive late for school. The relationships between sense of 
belonging and the three measures of literacy performance are also very weak, ranging from 0.04 to 0.06, 
The relationships between participation and academic performance are somewhat stronger, ranging from 
0.13 to 0. 14. 1 In contrast, the correlations among the three measures of literacy skills are fairly high, rang- 
ing from 0.68 to 0.79 at the student level. 


Figure 3.1 


Student-and school-level correlations among measures of engagement at school 
and performance on the reading, mathematical and scientific literacy scales 1 
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1. Only OECD countries are included. 


The correlation between sense of belonging and participation at the school level is 0.37, indicating a mod- 
erately strong relationship. Thus schools with high average levels of sense of belonging also tend to have 
high average levels of participation. The school-level correlations between the two engagement outcomes 
and the three measures of literacy performance are also moderately strong, ranging from 0.48 to 0. 51 . In 
contrast, the school-level correlations among the three measures of literacy performance are very strong, 
ranging from 0.97 to 0.99. 

These findings have a number of implications for educational policy and practice. The weak correlations at 
student level suggest that teachers and guidance counsellors are likely to encounter students who have a 
very low sense of belonging, even though they participate in school activities, and their literacy skills are 
fairly strong. Students with low participation are likely to have somewhat poorer literacy skills than those 
who have attended most classes; however, there are many students who miss school, skip classes and arrive 
late for school who also show reasonably strong literacy skills. 

The moderately strong school-level correlations among the engagement measures and literacy perform- 
ance suggest that schools that have high levels of engagement also tend to have high levels of academic per- 
formance. It cannot be infered from these findings that efforts to increase student engagement are likely 
to lead to better literacy skills. However, they do provide strong evidence that an emphasis on student 
engagement is seldom at the expense of literacy skill development, and vice versa: schools with low levels 
of student disaffection tend also to have strong literacy performance. 
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A typology of youth based on student engagement and literacy skills 

Another analytical approach to understanding the relationships among achievement and engagement is 
cluster analysis. Cluster analysis is commonly used to form clusters of individuals based on how similar 
they are with respect to a number of defining characteristics. The researcher decides which characteristics 
are most relevant. For example, if 100 students were being considered, the analysis would examine their 
characteristics, and in the first step form a cluster comprising the two students who were most similar. The 
next step would examine the characteristics of the 99 clusters (the newly formed cluster of two students 
and the 98 single student clusters) to find the two clusters that were most similar. It would join them into 
a new cluster. It would continue in this way, step by step, until a small number of clusters was formed that 
had relatively dissimilar characteristics. The researcher decides when to stop the process of joining clusters 
based on how dissimilar the clusters are at each step. 

Cluster analysis is used in this report to discern whether there are types of students based on their profile 
of engagement and literacy skills. The cluster analysis was performed for a sample of students comprising 
a random sample from each of the OECD countries. The analysis used the data describing students’ sense 
of belonging and participation, and their performance in reading and mathematical literacy. 2 


Figure 3.2 


Categories of students in OECD countries based on a cluster analysis of their sense of belonging, 
participation and reading and mathematical literacy performance 
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Figure 3.2 displays the results for the cluster analysis for the OECD student sample. In this analysis the 
clustering process was terminated at five clusters . The decision about where to stop the clustering process is 
somewhat arbitrary — any number of clusters can be chosen. However, after examining the results of numer- 
ous analyses, with attention to the composition and size of the clusters, five clusters seemed most appropri- 
ate. Figure 3.2 shows the percentage of students in each cluster, and the average scores on each of the four 
outcome variables for each cluster. These results are based on all students in participating OECD countries. 

The first cluster, which comprises about one-quarter of all students, is labelled top students. These stu- 
dents are engaged in schooling and have relatively high scores on reading and mathematical literacy. The 
second group, engaged students, have above average scores on the two engagement measures, hut on average 
have reading and mathematical literacy scores that are about 10 points below the OECD average of 500. 
Although these students do not tend to be among those with high literacy skills, they feel they belong at 
school and they are not absent from school on a regular basis. They also comprise about one-quarter of all 
students. The third group of students, labelled students Jeeling isolated, comprise about one-fifth of all stu- 
dents. These students on average have low scores on the sense of belonging scale, but above average levels 
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of participation. Their achievement scores tend to be fairly strong — on average about 20 points above the 
OECD average. The fourth group of students, labelled absentee students , has very low participation scores. 
Their literacy skills also tend to be below norms — on average about 50 points below the OECD average 
— but their sense of belonging is close to the OECD average. These students comprise about 10 per cent of 
the sample. The last group, labelled non- academic students, comprises students who have low literacy skills, 
on average about 1 30 to 135 points below the OECD average. These students on average have low scores 
on the sense of belonging scale, but are not absent from school on a regular basis. They comprise about 
17 per cent of the sample. 

An important finding revealed by this analysis is that students who have a low sense of belonging are found 
in two separate groups. There are students who feel lonely and isolated from their classmates, even though 
they have relatively high literacy skills. There are other students who have these feelings and have very poor 
academic performance. This split to some extent explains the relatively low correlations between sense 
of belonging and literacy skills (see Figure 3.1). An important issue concerning these results is whether 
students in the cluster with high literacy skills tend to pursue further education beyond the period of 
compulsory schooling. 

The cluster analysis also shows that students with very low literacy skills are not generally those with par- 
ticularly low scores on the two measures of engagement. The analysis did not yield a cluster of students 
who had low scores on all four outcome measures. 

In the next chapter, the relationship between the two measures of student disaffection and students’ char- 
acteristics, family background, and school factors are examined in detail. However, it is informative to 
observe the relationship between the prevalence of the five types of students with family socio-economic 
status. This study employed the same measure of socio-economic status as was used in Chapter 8 of Knowl- 
edge and Skills for Life (OECD, 2001a). It is a statistical composite of the educational level of the parents, 
the occupational status of the parents, the PISA indices of family wealth, home educational resources and 
home possessions related to classical culture. The gradients associated with each of the five types of students 
were determined by regressing each student type ( e.g . , a dummy variable denoting top student versus other) 
on socio-economic status and the square of socio-economic status. A logistic regression model was used, 
which is discussed in detail in the next chapter. The gradients are shown in Figure 3.3. 

In Figure 3.3, the student socio-economic status is standardised so that the average socio-economic status 
among the OECD country students becomes 0 and the standard deviation is 1 . The gradients indicate, 
for each of the five types of students, the expected prevalence, or percentage of students, associated with 
varying levels of socio-economic status. For example, the gradient for top students indicates that the likeli- 
hood of a student being in this category is strongly related with socio-economic status: the prevalence of 
top students among those of average socio-economic status (measured by the International Socio-Economic 
Index of Occupational Status and standardised to have an OECD average of 0 and a standard deviation of 1) is 
24.6 per cent, while the prevalence among those with a socio-economic status that is one standard devia- 
tion above the OECD average is 39.8 per cent. The gradient lines are drawn from the 5 th to the 95 th percentiles 
of socio-economic status for each group of students, thereby showing the range of socio-economic status for each 
group. The gradient line for top students ranges from -0.93 to 1 .73 on the scale for socio-economic status. 

The likelihood of a student being in the group labelled non-academic students is strongly negatively related to 
family socio-economic status. The range (from the 5 th to 95 th percentiles) is from -2.43 to 0.95. A student 
with average socio-economic status has a likelihood of 14.2 per cent of being among the non-academic 
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Fig u he 3.3 


Prevalence of students by socio-economic backgrounds, OECD countries 

% 



International Socio-Economic Index of Occupational Status (in standard deviation of OECD countries) 


Source: OECD PISA database, 2003. For data see www.pisa.oecd.org. 

group, whereas a student with a family socio-economic status that is one standard deviation below the 
OECD average is twice as likely — 28.4 per cent — of being in the non-academic group. 

In contrast, students in the other three categories tend to be from a wide range of family backgrounds, 
ranging from well below the OECD average to well above the OECD average. The prevalence of students 
in each of these three groups is not strongly related to socio-economic status. The gradients are markedly 
non-linear for the engaged and isolated groups, with students of average socio-economic status more likely 
to be in these groups than students with low or high socio-economic status. 

A separate cluster analysis was also performed for each country. The results are presented in Table 3. 3. The 
analyses revealed that the five-cluster model presented above fits for 25 of the 42 countries. However, in 
nine of the 42 countries, a solution based on four clusters provided a more inter pretable fit to the data, 
while in eight of the 42 countries, a three-cluster model was more appropriate. 

The countries with a four-cluster solution included Belgium, Finland, France, Iceland, Korea, the Nether- 
lands, Norway, Portugal and Spain. In these countries, there was a cluster of students who had relatively 
high scores on all four outcome measures. The cluster is referred to as well-rounded students. However, this 
cluster did not have the highest average literacy performance. A second cluster comprised students who 
had high average reading and mathematics performance, but tended to have a low sense of belonging. This 
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cluster was a fairly large cluster, comprising from 33.1 to 44.2 per cent of the students within each coun- 
try. The other two clusters had low average reading and mathematics performance. One cluster, which in 
all nine countries was the larger of the two, included youths who tended to have a low sense of belonging, 
while the other included youths who were regularly absent from school. As in other countries, there was 
no cluster of students with low average scores on all four outcomes. 

The three-cluster solution was used for eight countries: Bulgaria, Hungary, Indonesia, Israel, Italy, 
Luxembourg, the Russian Federation and Thailand. One of the clusters also comprised well-rounded 
students; these students, compared with other students in their country, had fairly high scores on all 
four outcomes. The other two clusters included a non-academic group of students who had a low sense 
of belonging, and a cluster of non-academic students who had fairly high scores on the two engagement 
measures, despite their low performance on the literacy measures. 

Conclusion 

This chapter examined the relationship between student engagement and literacy skills. A number of 
important findings emerged: 

(a) Students’ sense of belonging is not strongly' related to participation, or to any of the 
measures of literacy skills at the individual level. Many of the students who have a low 
sense of belonging tend to cluster into two groups, one that that has relatively high 
academic achievement, and another that has very low literacy' performance. Students 
with a low sense of belonging tend to have a wide range of socio-economic status. 
These results call for a conceptualisation of disaffection from school in the way that differs from how 
it has commonly been portrayed in the literature. Disaffection should not be simply thought of as an 
attitude that precedes and causes poor literacy skill development, but rather as a disposition towards 
schooling that is shared by youths from varying socio-economic backgrounds and with varying levels 
of literacy skills. These results indicate that poor literacy skill development is not the primary cause of 
disaffection. It may be that youth feel disaffected from school for a host of other reasons, such as their 
talent in sports, their personal appearance, or their ability to make friends easily. Further research 
could attempt to achieve a better understanding of when students become disaffected from school, the 
reasons underlying their disaffection, and how it is related to other academic and non-academic out- 
comes. It is important to under stand how students’ sense of belonging at school affects their decisions 
concerning post-secondary education. 

(b) Students’ participation is weakly related to measures ofliteracy performance. In many 
countries there is a small group of students who are absent from school on a regular 
basis and have achievement scores that are on average about 50 points below the OECD 
average. However, the majority of students who have very low literacy skills are not 
absent from school on a regular basis. About 10 per cent, or one-half of the students considered 
to have low participation, were in a cluster of students whose literacy performance was about one-half 
of a standard deviation below the OECD average, whereas those with very low literacy performance 
(more than 100 points below average) had participation scores that were close to the OECD average. 
More detailed analyses are required to understand why these students have low levels of participation. 
It may be that many of them have already decided upon a post-secondary destination that does not 
require high secondary school grades. If this is the case, they may be content with achieving passing 
grades, and therefore are more or less marking time until they have completed secondary school. 
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(c) Over one-half of all students are engaged at school and have strong or at least average 
literacy skills. The other half can be classified into three groups. One group, compris- 
ing about 20 per cent of all students, has a low sense of belonging, but fairly high lit- 
eracy skills. Another group, comprising about 10 per cent of all students, is regularly 
absent from school, and their academic literacy skills are below average. The remaining 
students are those with very poor literacy skills. The students in the latter group tend to come 
from families of lower socio-economic status. Although they have poor literacy skills, they are not 
strongly disaffected from school, nor are they regularly absent from school. The results of the previous 
chapter indicated that in most countries disengaged students are prevalent in most schools. Therefore, 
universal interventions, aimed at improving engagement for all students, may be more practical than 
targeted interventions focused on particular schools. These results, however, suggest that it may be 
practical to develop different kinds of school interventions aimed at helping students with particular 
types of problems. Youths who lack a sense of belonging at school are likely to need a different kind of 
intervention than those who are regularly absent. 

Taken together, these findings do not lend support to the popular belief that efforts to increase student 
engagement will result in large gains in literacy skills. This may he the case for a small proportion of stu- 
dents who are regularly absent from school and are performing slightly below norms in their academic 
work. Rather, the findings indicate that there are many students who are disaffected from school who 
are doing well academically, and vice versa. Engagement should not thus he viewed simply as a precur- 
sor to academic achievement and treated as a control variable in monitoring activities and in educational 
research. Engagement is an important outcome in its own right, as there are many students who lack a 
sense of belonging at school, and do not participate fully in school activities. These students would likely 
benefit from school policies and practices that increased their sense of belonging and participation. 


Notes 

1 . These correlations are weaker than those reported in the literature based on smaller- scale studies conducted in the United 
States (eg., Goodenow, 1993; Voelkl, 1995). However, the estimates in this study for the United States, which range from 
0.12 to 0. 1 5 for sense of belonging, and 0. 19 to 0.22 for participation (see Table 3.1), are more consistent with the previous 
literature. 

2. Science performance was not used in these analyses because reading and science are highly correlated at the student level. 
Preliminary analyses indicated that the inclusion of science performance did not appreciably affect the results of the cluster 
analysis. 
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Introduction 

The aim of this chapter is to identify the principal family and school factors related to student disaffection. The 
outcome variables in these analyses are the measures used earlier that classified students as either having or not 
having a low sense of belonging or low participation. The first set of analyses examines the relationships between 
the engagement measures and the students’ gender, socio-economic status, whether they were foreign-born, 
and whether they were living in a one-parent or two-parent family. These analyses are conducted for all OECD 
countries together and then separately by country. The second set of analyses estimates the effects associated 
with a number of factors describing the schools students were attending. These factors are categorised into three 
groups: school context, school resources, and school policy and practice. The results of these analyses call for a 
closer examination of the variation among countries in some of the key predictors of student disaffection. 


Box 4.1 Logistic regression and odds ratios 

Multiple linear regression is generally used to express the relationship between an outcome (or 
dependent) variable, such as students’ reading performance, and a set of predictor (or independent) 
variables, such as family income, parents’ education, and the sex of the child. The researcher posits 
a model of the relationship between the dependent and independent variables, which is usually 
based on some theory or on prior research. The data are Jit to the model, yielding a set of weights 
called regression coefficients. These denote the expected change inY (the dependent variable) Jot a one- 
unit change in X (the independent variable), given all other variables in the model are held constant. Multiple 
regression is a powerful tool for policy purposes, because the effects of a policy variable of interest 
(i e.g . , the effects of the classroom pupil-teacher ratio on reading performance) can be examined, 
while holding other variables constant (e.g. , the family background of students). 

Multiple regression is appropriate when the outcome variables are continuous, such as the measures 
of reading, mathematics, and science performance used in PISA. However, when the outcome vari- 
able is dichotomous, such as whether or not a child repeated a grade at school, a variant of multiple 
regression called logistic regression is appropriate. It is also a useful policy device, because often 
the interest is in the prevalence of certain traits, such as children with disabilities, or in this study, 
youth considered disaffected from school or regularly absent. The policy analyst is interested in the 
probability or likelihood of the child having the trait, or experiencing the event at a particular time, 
and how various characteristics of the child, such as age, sex, or family income, or a characteristic 
of the school, such as pupil-teacher ratio, affect that probability. The regression coefficients from 
a logistic regression can be easily transformed to odds ratios, which can be simply interpreted for 
policy purposes. 

The odds of an event occurring is the likelihood of the event occurring divided by the likelihood of the 
event not occurring. For example, if an event has a 75 per cent chance of occurring, then the odds 
of it occurring are [0.75/ (1-0.75)], which is 3.0. An event with an odds of 1 .0 has an equal chance 
of occurring or not occurring. An odds ratio is simply the ratio of the odds for two different sets of 
circumstances. For example, the odds of an event occurring for girls and for boys could be assessed, 
and the ratio of the odds could be calculated. Odds ratios are interpreted in a fashion similar to mul- 
tiple regression coefficients: they denote the ratio of the odds of an event occurring after a one-unit 
change in the independent variable, compared to what it was previously, given all other independent 
variables in the model are held constant. 
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In Figure 4.1, for example, the results from two logistic regression analyses are presented. In the 
first analysis, the outcome variable is whether or not a student had low sense of belonging. One 
of the independent or predictor variables is whether or not the student was born in the country. 
The odds ratio is 1 .37. This indicates that the likelihood of a youth who was foreign-born being 
disaffected is 1 .37 times that of a youth who was born in the country. The predictors for a logistic 
regression can include continuous variables, such as a child’s age, or a continuous measure of socio- 
economic status. In Figure 4.4, for example, the analysis includes a number of variables describing 
school resources, and classroom and school policy and practice. The odds ratio for low participation 
associated with disciplinary climate is 0.95. This indicates that the odds of a child displaying low 
participation is about 5 per cent less for each one-point increase on the scale for disciplinary climate. 
Norusis/SPSS Inc. (1992) provides a simple introduction to logistic regression. 


The analyses in this chapter use a regression technique called logistic regression. This technique is 
appropriate when the outcome variable for a regression analysis is dichotomous, such as whether or not a 
child has a low sense of belonging. In this case, the outcome denotes whether or not a student has a low 
sense of belonging or low participation. What is of interest is the probability or likelihood of the student 
being disaffected, and how various characteristics of the student, such as age, sex, or family income, or 
characteristics of the schools they attend, affect that probability. Also, because the data are hierarchically 
nested (students nested within schools, and schools within countries), the appropriate model is a 
multilevel logistic regression. This adds some complexity to the analysis, but for the non- statistical reader, 
the interpretation is reasonably straightforward. Box 4. 1 provides a discussion of logistic regression. 

The relationship between student engagement and family background 

Figure 4. 1 displays the odds ratios for five variables describing students’ characteristics and family back- 
ground. These results are based on a three-level multilevel logistic regression model, with students nested 
within schools, and schools nest ed within countries. The results indicate that the odds of a female having 
a low sense of belonging are 0.98 that of a male. This is close to 1 .0, and not statistically significant, indi- 
cating that males and females are equally likely of having a low sense of belonging. For low participation, 
however, the odds ratio is 0.93, which suggests that the odds of a female having low participation is about 
7 per cent less than that of males. This difference is statistically significant. The results also indicate that 
for low sense of belonging, the odds ratios for gender vary significantly among schools within countries, 
and among countries. This calls for further analyses that examine gender differences separately for each 
country. Country-level results are reported in Table 4.1. 

Living in a family of high socio-economic status (top quartile within the country) offers some protective 
effect from having a low sense of belonging. The odds ratios is 0. 86, which indicates that the odds of having 
a low sense of belonging for a student living in a high socio-economic status family is about 14 per cent less 
than the odds for students living in families of average (middle two quartiles) socio-economic status. The 
odds ratio for low participation is 0.94, and not statistically significant. 

The deleterious effects on students’ engagement associated with living in a family of low socio-economic 
status (bottom quartile within the country) are considerably greater. The odds of having a low sense of 
belonging are about 38 per cent greater for students living in low socio-economic status families than for 
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Figure 4.1 


Student characteristics and family background factors associated with low sense of belonging or low 
participation (logistic regression coefficients, standard errors and odds ratios) 1 


Low sense of belon 

ging 

Low participation 


Effect 

Standard error 

Odds ratios 2 

Effect 

Standard error 

Odds ratios 2 

Female student 

-0.018 

(0.03) 

0.98" 

-0.076 

(0.03) 


0.93 c 

High socio-economic family background 

-0.157 

(0.02) 

0.86' 

-0.063 

(0.03) 


0.9+ c 

Low socio-economic family background 

0.32+ 

(0.02) 

1.38 c 

0.234 

(0.0+) 


1.26 c 

Foreign-born student 

0.313 

(0.03) 

1.37' 

0.261 

(0.06) 


1.30' 

Single -parent family 

0.159 

(0.01) 

1.17 

0.302 

(0.02) 


1.35 


1. Only OECD countries are included. 

2. Odds ratios in bold text are statistically significant (p < 0.05). Odds ratios with a superscript s vary significantly among schools within coiuitries, and 
those with a superscript c vary significantly among countries. 


those living in average socio-economic status families. Low socio-economic status is also a risk factor for 
low participation; the odds ratio is 1.26. 

Students who were foreign-born 1 are also at greater risk of having low sense of belonging or low 
participation. The odds ratios are comparable to those associated with low socio-economic status: 1.37 
for low sense of belonging and 1 . 30 for low participation. 

Students living in single-parent families - are also more likely to be disaffected from school: the odds ratio 
for low sense of belonging is 1.17 and for low participation is 1 .35. 

Many students have multiple risk factors. For example, many students who have experienced immigration 
to a new country also live in low socio-economic families, hi most countries, a disproportionate number 
of youths living in single parent families are also of low socio-economic, status. Odds ratios indicate the 
effects associated with each predictor variable, given that all other variables in the model are held constant. 
Odds ratios can be multiplied. Thus, the odds of being disaffected for a youth living in a single-parent, low 
socio-economic status family is about (1.38 * 1.17 = ) 1.61 limes that of a youth living in a two-parent 
family of average socio-economic status. 

The results indicate that the protective effects of living in a high socio-economic status family and the risks 
associated with being foreign-born or living in a low socio-economic status family vary significantly among 
countries, but not among schools within countries. These results emphasise the need for examining these 
effects for each country separately. 

Estimates of the odds ratios for low sense of belonging associated with students’ characteristics and family 
background are reported in Table 4. 1. The results for low participation are displayed in Table 4.2. The 
results displayed in Figures 4. 1 to 4.3 reveal that living in a low socio-economic, status family is one of the 
most important risk factors for student disaffection and low participation. 

hi seven OECD countries — Hungary, Iceland, Luxembourg, Norway, Poland, Portugal and the United 
States — and in four non- OECD countries — Argentina, Bulgaria, Latvia and the Russian Federation — stu- 
dents with low socio-economic status backgrounds were more than 50 per cent more likely than their 
peers with average socio-economic status backgrounds to have a low sense of belonging. In all but three 
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countries — Brazil, Finland and Liechtenstein — the odds ratios exceeded 1.20, stressing that in virtually 
every country students from poor families are more likely to feel lonely or feel like an outsider at school. 

Generally students from affluent families were less likely to experience these feelings, but the effect was 
less pronounced. Living in a high socio-economic status family was a significant protective factor in 14 of 
the 28 OECD countries, and in 10 of the 14 non-OECD countries. In 17 of those 24 countries, the odds 
ratio was 0.80 or lower, indicating that the odds of low sense of belonging for students with high socio- 
economic backgrounds were at least 20 per cent less than for their peers with average socio-economic 
backgrounds. 


Figure 4.2 


Student characteristics and family background factors associated with low sense of belonging 


■ Students from low socio-economic status families ■ Students from High socio-economic status families 



1 . Response rate is too low to ensure comparability. 
Source: OECD PISA database, 2003. Table 4. 1 . 


Being a foreign-born student is also an important risk factor for low sense of belonging (seeTable 4.1). On 
average, across all participating OECD countries, the effect is of the same magnitude as the effect of living 
in a low socio-economic family. Considering that foreign-born students in many countries tend to also 
work in low status occupations, the effects are compounded for many students. The odds associated with 
being foreign-born vary considerably among countries. In 14 of the 28 OECD countries, and in two of the 
14 non-OECD countries, the odds ratios were statistically significant. In most of these countries, the odds 
ratios exceeded 1 .50, indicating that the odds of low sense of belonging for foreign-born students in these 
countries were at least one and a half times as great as those for students who were born in the country. 
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Students in single-parent families were significantly more likely to have a low sense of belonging in 9 
OECD countries, and *4- non-OECD countries. However, on average the magnitude of the effects was 
not as large as those associated with being foreign-born or living in a low socio-economic family (see 
Table 4. 1). 


Figure 4-. 3 


Student characteristics and family background factors associated with low participation 


■ Students from low socio-economic status families ■ Students from high socio-economic status families 



Source: OECD PISA database, 2003. Table 4.2. 


Table 4.2 shows the country-level results for low participation. In nine of the OECD countries — Belgium, 
Iceland, Ireland, Hungary, Japan, New Zealand, Sweden, the United Kingdom and the United States — the 
odds ratio for living in a low socio-economic family was at least 1 . 50. These results indicate that the odds 
of low participation for students from low socio-economic families in these countries were at least one and 
a half times as great as the odds for students from average socio-economic status families. In non-OECD 
countries, living in a low socio-economic status family was a less important risk factor for low participa- 
tion. In all countries the odds ratios were less than 1.25. 

Similar to the results for low sense of belonging, the protective effect of living in a high socio-economic 
status family is smaller than the deleterious effect of living in a low socio-economic status family. On aver- 
age, the risk associated with living in a single parent family is slightly greater than the risk of living in a low 
socio-economic status family. The odds ratio for single-parent families was 1 . 50 or greater in eight of the 
OECD countries and in one non-OECD country. 
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In seven of the 28 OECD countries, and 10 of the 14 non-OECD countries, females were significantly 
less likely than males to have low participation (see Table 4.2). However, in three — Liechtenstein, New 
Zealand and Spain — females were more likely than males to have low participation. On average, across the 
OECD countries, the odds of low participation for females was about 93 per cent that of males. 

The results pertaining to foreign-born students are especially interesting because they vary considerably 
among countries. On average, for the 28 OECD countries, the odds ratio was 1.30. However, in most 
OECD countries, low participation was not significantly related to whether the youth was foreign-born, 
while in four countries — Belgium, France, Norway and Switzerland — the odds ratios ranged from 1.51 
to 2.00. Non-OECD countries also vary considerably in their results for foreign-born students. It is a 
significant risk factor in only two countries — Indonesia and Liechtenstein. In Israel, the prevalence of low 
participation was lower among foreign-born students than among those born in the country. 

The effects of family and school factors on levels of student engagement 

In the next set of analyses, the model presented above is extended to include variables describing school 
context, school resources, and school policy and practice. One of the important findings presented in 
Knowledge and Skills for Life (OECD, 2001a) was that the average level of socio-economic status in a school 
was associated with higher student achievement. In nearly every country, the effect was large and statisti- 
cally significant, indicating that a student with average family background characteristics tended to have 
higher achievement if he or she attended a school with high socio-economic levels. This effect tended to 
be slightly stronger for males and students from low socio-economic backgrounds. Consequently, when 
students are segregated along socio-economic lines, students from advantaged backgrounds fare slightly 
better in their literacy skills, while those from less advantaged backgrounds fare considerably worse. The 
analyses in the first international report also pointed to a number of important factors regarding school 
policy and practice, and concluded that: 

“ . . . from those factors that were examined in PISA, there is no single factor that explains why some 
schools or some countries have better results than others. Rather, successful performance is attribut- 
able to a constellation of factors that includes school resources, school policy and practices and class- 
room practices.” (OECD, 2001, p. 212) 

For student performance in reading, the most prominent factors included: lower student-teacher ratios 
(at least below 25), larger schools (at least up to 1,000 students), a high proportion of teachers with spe- 
cialised training in their subject domain, students’ use of school resources ( e.g . , school library, computers, 
calculators, laboratories and the internet via the school), school climate, teacher morale and commit- 
ment, school autonomy, student-teacher relations, disciplinary climate of the classroom, and academic 
press, which is a measure of the extent to which teachers have high expectations for student success and 
emphasise academic performance. These school-level measures are described in Knowledge and Skills for Life 
(OECD, 2001a). 

The results for low sense of belonging and low participation are presented in Figure 4.4. The first model 
in these analyses includes the set of variables describing student characteristics and family background, 
plus two measures of school context: the percentage of students from high socio-economic families and 
the percentage of students from low socio-economic families. The results indicate large and significant 
effects associated with school context. Students who are attending schools with a higher percentage of 
students from low socio-economic status families are more likely to have a low sense of belonging and 
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low participation than those attending schools with few students from low socio-economic status family. 
An increase of 10 per cent in the percentage of students from low socio-economic status families in the 
school is associated with a 5 per cent increase in the odds of a student having a low sense of belonging, and 
a 7 per cent increase in the odds of low participation. The percentage of students in the school living in 
high socio-economic status families was not significantly related to either a low sense of belonging or low 
participation. 


Figure 4.4 


Family and school factors associated with a low sense of belonging or low participation 
(odds ratios), OECD countries 


Family and school factors 
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1.01" 
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Single -parent family 
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1.17 
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School context 








Percentage of students from high socio-economic families 
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1.00* 
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Percentage of students from low socio-economic families 
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School size 
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0.98 
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1.00 
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0.99 
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School policy and practice 
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0.99 
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0.95 
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0.98 
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0.95* 

0.95* 

Achievement press (10-point index) 
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0.99 c 

0.99* 
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0.97* 


Source: OECD PISA database 2003. 


The second model includes independent variables describing student characteristics and family back- 
ground, and measures describing school resources together with school policy and practice. In this model, 
as with those presented earlier, the odds ratio for each independent variable denotes the ratio of the odds 
of an event occurring after a one-unit change in the independent variable, compared to what it was previ- 
ously, given that all other independent variables in the model are held constant. The measures of school 
resources, school policy and practice were derived as follows: 
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School size was derived from the school administrators’ report of the school enrolment. One unit on this 
scale represents 100 students. 

Student-teaching staff ratio was defined as the number of full-time equivalent teachers divided by the number 
of students in the school. One unit on this variable represents a change of one student per teacher. 

Quality of school infrastructure is a summary measure derived from school principals’ reports of the extent 
to which the learning of 15-year-olds was hindered by (a) poor condition of buildings; (b) poor heating, 
cooling and/ or lighting systems; (c) lack of instructional space ( e.g ., classrooms); (d) lack of instructional 
material (e.g., textbooks); (e) not enough computers for instruction; f) lack of instructional materials 
in the library; and (g) inadequate science laboratory equipment. One unit on this scale represents 
10 percentile points, with higher scores indicating a better quality of school infrastructure. 

School administrators’ assessment of teaching staff was derived from school principals’ reports of the extent 
to which the learning of 15-year-olds was hindered by: (a) low expectations of teachers; (b) poor stu- 
dent-teacher relations; (c) teacher turnover; (d) teachers not meeting individual student needs; (e) teacher 
absenteeism; f) staff resisting change; (g) teachers being too strict with students; and (h) students not being 
encouraged to achieve their full potential. One unit on this scale represents 10 percentile points, with 
higher scores indicating more positive ratings of teaching staff. 

Use offormal assessment was derived from school principals’ reports on the frequency with which standardised 
tests were used, and on whether or not the assessments were used to monitor the school’s progress from 
year to year and monitor the school’s progress from year to year. One emit on this scale represents 
10 percentile points, with higher scores indicating greater use offormal assessments. 

Teacher morale and commitment was derived from school principals’ reports on the extent to which they 
agreed with these statements concerning teacher morale and commitment: (a) the morale of teachers in 
this school is high; (b) teachers work with enthusiasm; (c) teachers take pride in this school; and (d) teachers 
value academic achievement. One unit on this scale represents 10 percentile points, with higher scores 
indicating a higher level of teacher morale and commitment. 

Teacher autonomy was derived from a question asked of principals as to who had the main responsibility for: 
(a) hiring teachers; (b) firing teachers; (c) establishing teachers’ starting salaries; (d) determining teachers’ 
salary increases; (e) formulating the school budget; ff) deciding on budget allocations within the school; 
(g) establishing student disciplinary policies; (h) establishing student assessment policies; (i) approving 
students for admittance to school ; (j ) choosing which textbooks are used ; (k ) determining course content ; and 
(1) deciding which courses are offered. This scale indicates the extent to which teachers had responsibility 
for these activities. One unit on this scale represents 10 percentile points, with higher scores indicating a 
higher level of teacher autonomy. 

School autonomy was derived from the same question described above. In this case, the scale indicates the 
extent to which principals had responsibility for the various activities. One unit on this scale represents 
10 percentile points, with higher scores indicating a higher level of principal autonomy. 

Use of irformal assessment was derived from school principals’ reports on the frequency with which students 
were assessed using teacher-developed tests, teachers’ judgemental ratings, student portfolios and student 
assignments/projects/homework, and on how frequently assessment information was formally commu- 
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nicated to parents and the school principal. One unit on this scale represents 10 percentile points, with 
higher scores indicating greater use of informal assessments. 

Student-teacher relations was based on students’ reports of the extent to which they agreed or disagreed with 
the following statements concerning student-teacher relations: (a) students get along well with teachers; 
(b) most teachers are interested in students’ wellbeing; (c) most of my teachers really listen to what I have 
to say; (d) if I need extra help, I will receive it from my teachers; and (e) most of my teachers treat me fairly. 
The student scores were aggregated to the school level, and scaled such that one unit on the scale repre- 
sents 10 percentile points, with higher scores indicating better student- teacher relations. 

Disciplinary climate was based on students’ reports of the extent to which they agreed or disagreed with the 
following statements concerning student-teacher relations: (a) the teacher has to wait a long time for stu- 
dents to quieten down; (b) students cannot work well; (c) students don’t listen to what the teacher says; (d) 
students don’t start working for a long time after the lesson begins; and (e) there is noise and disorder. The 
student scores were aggregated to the school level, and scaled such that one unit on the scale represents 10 
percentile points, with higher scores indicating a more positive disciplinary climate. 

Achievement press was based on students’ reports of the extent to which they agreed or disagreed with the 
following statements concerning teachers’ expectations: (a) the teacher wants students to work hard; 
(b) the teacher does not like it when students deliver careless work; (c) the teacher checks students’ 
homework; and (d) students have a lot to learn. The student scores were aggregated to the school level, 
and scaled such that one unit on the scale represents 10 percentile points, with higher scores indicating 
greater press for academic achievement. 

For the first variable, school size, one unit represents 100 students. Thus the odds ratio for low sense of 
belonging, 0.98, indicates that the odds of a student having a low sense of belonging decreases by about 
2 per cent for a 100 -student increase in school size. For the student-staff teaching ratio, the odds ratio indi- 
cates the ratio associated with a one-student change in the student-staff teaching ratio. For low participation, 
for example, a one-student increase in the student-staff teaching ratio is associated with a 3 per cent decrease 
in the odds of a student having low participation. For both these variables, a squared term is also included. The 
analysis indicated that some of these relationships were curvilinear; for example, in the case of school size for 
low sense of belonging, the relationship becomes weaker with increasing levels of school size. 

The other variables describing school resources, policy, and practice were scaled on a 10-point scale, such 
that the results pertain to an increase or decrease in the odds associated with a one-point increase on the 
10-point scale. They were scaled such that a school that scored 3.4 on this scale, for example, would be 
at the 34 th percentile among all OECD schools participating in the survey. Similarly, a school with a score 
of 7.5 would be at the 75 th percentile. This helps to assess the importance of the effects of these variables 
in an easily interpreted metric, which can be compared across the policy variables. For example, for low 
sense of belonging, the odds ratio for student- teacher relations is 0.95. This indicates that a one-point 
increase on the 10-point scale, which represents, for example, the difference between a school at the 
50 th percentile in student-teacher relations versus a school at the 60 th percentile, is associated with about a 
5 per cent decrease in the odds of a student having a low sense of belonging. 

The models fit to the full data set could be fit separately for each country. However, the results for many 
countries are not very stable, as the number of schools sampled in each country is rather small for estimat- 
ing models with a large number of school variables. Therefore, the strategy is to use the full international 
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data set to determine the effects associated with each of the policy variables, and then examine how coun- 
tries fare on the most relevant variables. 

Three school-level factors emerged as having a significant impact on students’ sense of belonging. Students 
in larger schools were less likely to have a low sense of belonging; however, the effect is small: an increase 
of 100 students is associated with a decrease in the odds of about 2 per cent. The model also included a 
term for the square of school size. Its effect was also statistically significant, and slightly above 1.0, indicat- 
ing that the benefits associated with school size level off with increasing school size. However, this effect is 
also very small. The more important effects for low sense of belonging pertain to students’ assessments of 
student -teacher relations and the disciplinary climate of the classroom, which respectively were associated 
with 5 per cent and 2 per cent decreases in the odds for each one-point increase on the 10-point scales. 

Considering the models for low participation, five of the 1 2 school-level factors emerged as statistically 
significant predictors. Larger schools tend to have a slightly higher prevalence of low participation stu- 
dents, but the relationship is fairly weak: a 100-student increase is associated with a 2 per cent increase in 
the odds. A one-student increase in the student-staff teaching ratio is associated with a 3 per cent decrease 
in the odds of a low participation. This is contrary to expectations, as one would expect higher levels of 
participation in smaller classes. The most important effects were associated with student-teacher relations, 
disciplinary climate and achievement press. For these factors, a one-point increase on the 10-point scale 
was associated with decreases in the odds, respectively, of 4 per cent, 5 per cent and 3 per cent. 

The final model, Model III, includes all of the factors discussed above. This model was estimated to dis- 
cern whether some of the contextual effects associated with living in either a low or high socio-economic 
background were attributable to school resources, policy and practice. Recall that the odds ratios denote 
the ratio of the odds of an event occurring after a one-unit change in the independent variable, compared 
to what it was previously, given that all other independent variables in the model are held constant. Findings in 
Knowledge and Skills for Life (OECD, 2001, see Table 8.6) showed that certain aspects of school resources, 
policy and practice are correlated with school context. For example, in many countries schools with a high 
average socio-economic status tend to have more positive student-teacher relations and greater achieve- 
ment press. Therefore, these classroom factors could be expected to explain some of the effects of school 
context on engagement, and vice versa. Kraemer et al. (2001) provide a useful discussion of mediating and 
overlapping factors. 

In the full model for low sense of belonging, the odds ratios do not change substantially from those dis- 
cussed above. The strength of the contextual effect associated with high socio-economic families is slightly 
stronger, and statistically significant, while the strength of the contextual effect associated with low socio- 
economic families is slightly weaker. The contextual effect associated with high socio-economic families is 
contrary to expectations. This is the estimated effect after controlling for all of the other variables in the 
model, which may be slightly greater than 1 .0 because the percentage of students of high socio-economic 
status is correlated with some of the school process variables. In any event, the effect is very weak, and not 
important in substantive terms. 

Similarly, the odds ratios for low participation differ only slightly from the estimates for Models I and II. The odds 
ratios for the contextual effects are slightly weaker, suggesting that some of the contextual effect is attributable 
to school resources, policy and practice. However, the estimated effects associated with school resources, policy 
and practice were virtually unchanged. Thus, contextual effects and the three school policy and practice factors 
— student -teacher relations, disciplinary climate and achievement press — remain the most important variables. 
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Figure 4.5 shows the mean scores on the three school policy and practice variables which were found 
to be important predictors of disaffection from school. These measures have been scaled such that 5.0 is 
the average score for all OECD schools. The scores indicate how a country’s schools fare in terms of the 
OECD distribution of scores. For example, the score of 7.0 for Japan indicates that the average school in 
Japan would rank at about the 70 th percentile on the OECD distribution. 

On the measure of disciplinary climate there are five OECD countries with relatively high average scores, 
that is, at or above 6.0 or the 60 th percentile. These are: Austria, Japan, Mexico, Poland and Switzerland. 
Seven of the non-OECD countries had similarly high levels on this index: Albania, Indonesia, Latvia, 
Liechtenstein, FYR Macedonia, the Russian Federation and Thailand. There are five OECD countries 
with relatively low scores on this measure, below 4.0: Greece, Italy, the Netherlands, Norway and Sweden. 
Among the non- OECD countries, four had scores that were below 4.0: Argentina, Brazil, Chile and Israel. 

Ten of the OECD countries had relatively high scores, above 6.0, on the measure of teacher-student rela- 
tions: Australia, Canada, Denmark, New Zealand, the Netherlands, Portugal, Sweden, Switzerland, the 
United Kingdom and the United States. Only two non-OECD countries, Brazil and Thailand, had scores 
above 6.0 on this measure. Five of the OECD countries — Austria, Germany, Japan, Luxembourg and 
Poland — had scores below 4.0, while none of the non-OECD countries were at this level. These results 
indicate that there is considerable variability among countries in their levels of student-teacher relations. 

The average scores for achievement press also vary substantially among countries. Eight OECD countries 
had scores above 6.0: Canada, LIungary, Iceland, Ireland, Italy, New Zealand, the United Kingdom and 
the United States. Three of the non-OECD countries met these criteria: Hong Kong-China, Latvia and 
the Russian Federation. In contrast, seven OECD countries scored below 4.0: Austria, Belgium, the 
Czech Republic., Greece, Japan, Korea, Luxembourg and the Netherlands. Five non-OECD countries — 
Argentina, Bulgaria, Indonesia, Israel and FYR Macedonia — scored below 4.0 on this measure. 

As mentioned earlier, the strategy used in this last set of analyses was to use the full international data set 
for participating OECD countries and try to discern which family and school variables have the greatest 
impact on student engagement. Individual countries can then assess how well they are performing on the 
most important indicators. The limitation to this approach is that the effects associated with school factors 
such as disciplinary climate, student-teacher relations, or achievement press, may he strong in some coun- 
tries but not in others. The school-level analyses should be conducted separately within each country, but, 
as mentioned earlier, the statistical power of the analyses would be weak for most participating countries. 
Thus, a very large sample of schools is needed to get some purchase on the potential effects of school-level 
variables. Although this analysis was successful in identifying some of the key aspects of schooling that 
affect student engagement at a very macro level, there are likely to be other schooling factors that affect 
engagement within particular countries. 
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Figure 4.5 


Mean indices of disciplinary climate, student-teacher relations and achievement press 



1 . Response rate is too low to ensure comparability. 
Source: OECD PISA database, 2003. Table 4.3. 
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Conclusion 

This chapter examined the relationships between student engagement and family and school factors. Sev- 
eral of the findings are particularly relevant to educators and school administrators: 

(a) On average, females were equally as likely as males to have a low sense of belonging, 
but were about 7 per cent less likely to exhibit low participation. However, gender 
differences for both aspects of engagement varied considerably among countries. The 

measure of students’ sense of belonging included questions about whether students feel accepted at 
school, make friends easily or feel like an outsider. It might be expected that females would be as likely 
as males to experience such feelings, which may be affected more by the nature of the peer group in a 
school than by family or school processes. 

(b) Students born outside the country are more likely than other youths to have a low 
sense of belonging. This finding points to the importance of classroom and school policies and prac- 
tices that teachers and principals can implement to promote social inclusion (Cuban, 1989; Reichl, 
2000). The findings also indicate that students are more likely to be disaffected from school if they are 
in low socio-economic schools. The results did not suggest that this contextual effect differed for for- 
eign-born students than for those born in the country. Taken together, therefore, these findings suggest 
that it is important to ensure that foreign-born youths are not disproportionately represented in low 
socio-economic status schools. 

(c) Students from low socio-economic families are more likely to be disaffected from 
school, as are students who attend schools that have a high percentage of students of 
low socio-economic status. As these risk factors compound, students from low socio- 
economic status families are even more likely to be disaffected from school. This phe- 
nomenon of double jeopardy (Willms, 2002 b) is also evident in analyses of student achievement: low 
socio-economic status students who also attend schools that predominantly serve low socio-economic 
status students are especially at risk of poor school performance because they have two factors working 
against them. 

Students are more likely to be engaged at school if they attend schools that have a high average socio-eco- 
nomic status, a strong disciplinary climate, good student-teacher relations and high expectations for stu- 
dent success. This study found that measures of engagement were correlated with these school factors. The 
discussion at the beginning of this report emphasised that student engagement begins early in a child’s life. 
However, the data describing school resources and school policy and practice in this study reflect students’ 
and administrators’ most recent experiences. Therefore, it is likely that the observed effects on engage- 
ment would be even stronger if students could be tracked over several years, with measures of engagement 
alongside comparably strong measures of school and classroom processes. Also, a point that should not be 
overlooked is that the most important factors affecting engagement are not necessarily the most expensive. 
For example, the results suggest that engagement is related to factors associated with classroom practice, 
yet unrelated to the quality of the school infrastructure. 
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Notes 


1 . Students were considered foreign-born if they were not born in the country where they were tested. This measure does not 
take into account how long they may have been in the country, or the ethnic origin of their parents. 

2. Families were considered to be single-parent families if they were headed by any one of: the mother, the father, a female 
guardian, or a male guardian. 
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fl CHAPTER 5 Student Engagement at School: A Sense of Belonging and Participation 

Introduction 

One of the primary aims of the OECD PISA is to provide a richer understanding of how well youths are 
prepared to meet the challenges they will encounter after finishing secondary school. PISA not only meas- 
ures the broad set of knowledge and skills that young people will need as they enter the knowledge society, 
it also assesses how well students can apply these skills in real-life situations. This report examines student 
engagement at school, which is viewed theoretically as a 11 disposition towards learning, working with others 
and functioning in a social institution”. PISA offers an opportunity to study student engagement because 
it has collected reliable and valid data on students’ literacy skills at age 15 alongside detailed information 
on students’ attitudes and values, their family background and the social and organisational structures of 
the schools they attend. Moreover, the sheer magnitude of PISA, with data for 224,058 students in 8, 564 
schools, acToss 42 countries, provides a means for estimating relationships between engagement and family 
and school factors that can be generalised to the population of 15 -year-old students in each country and 
compared cross-nationally. 

Student engagement refers to whether students feel they belong at school, accept the broader societal 
values associated with schooling and participate in school activities. PISA includes two measures of engage- 
ment. One pertains to students’ sense of belonging — whether students feel accepted at school, as opposed 
to feeling lonely or rejected. The other pertains to student attendance, which in most studies of engage- 
ment is considered the primary indicator of participation. These two measures were standardised to have a 
mean of 500 and a standard deviation of 100, similar to the measures of reading, mathematics and science 
performance, hi addition, two benchmarks were constructed that specified whether or not students had 
particularly low scores on the sense of belonging or participation scales. 

Although PISA is in many respects effective in studying student engagement, it also has some limitations. 
The chief limitation is that the data are cross-sectional; that is, collected from the respondents on only one 
occasion. Students’ level of engagement undoubtedly varies with age. An important question is whether 
there are critical periods when students develop an attachment to school, or whether it is a disposition 
that is cultivated continuously by families, peer groups and schools. A fuller under standing of engage- 
ment could be achieved by a study that followed students from an early age. Longitudinal data would also 
provide a better purchase on how aspects of school environments affect student engagement and whether 
there is a causal link between the acquisition of literacy skills and engagement. 

hi Chapter 2, some of the issues concerning the measurement of students’ sense of belonging and partici- 
pation were discussed. Because PISA is a multi-purpose study, aimed at meeting the policy-driven interests 
of many countries, it is not possible to measure all constructs in detail. Also, these constructs have a cul- 
tural component and a local context that cannot be fully captured in a large international study. Further 
research within countries could examine the nature of student engagement in schools and the role that 
parents and educators could play in bringing about greater participation and a stronger sense of belonging. 
Smaller, more focused studies of student engagement could include questions that assessed the extent to 
which students value school success and view it as important to their future. 

The data collected in PISA furnish reliable estimates of the average levels of sense of belonging and par- 
ticipation at country level. They also provide reliable estimates, across and within countries, of the rela- 
tionships between the engagement measures and a broad set of factors describing family background. The 
data are also sufficient for estimating the overall relationships between engagement and school factors for 
all countries. However, the number of schools sampled in most countries is not sufficient to make strong 
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claims about the relationships between measures of engagement and school-level factors at the country 
level. Because of this limitation, country-by-country estimates of the potential impact of school-level fac- 
tors can only be considered indicative. More accurate estimates could be garnered by taking larger samples 
of schools, while maintaining within-school sample sizes, but this is costly. When data become available 
for future cycles of PISA, it may be possible to combine the data across several cycles to achieve more 
accurate estimates of the relationship between engagement and school-level factors. Future analyses can 
also examine whether changes in countries’ school policies and practices are related to changes in levels 
of student engagement. 

Key findings 

An important finding of this study is that students’ reports of their sense of belonging and 
absenteeism indicate that there is a high prevalence of students who are disaffected from 
school. Among OECD countries, about 25 per cent of all students were considered to have a low sense of 
belonging, and about 20 per cent were regularly absent from school. The prevalence of both low sense of 
belonging and low participation was even higher among non-OECD countries. Moreover, the prevalence 
of low sense of belonging was high in all countries, ranging from 17 per cent to 41 per cent among OECD 
countries, and from 17 to 40 per cent among non-OECD countries. The prevalence of low participation 
varied more among countries, ranging from 4 to 34 per cent among OECD countries, and from 3 to 
45 per cent among non-OECD countries. 

However, the emphasis of this study on student disaffection diverts attention from the 
fact that most students are not disaffected from school. Indeed, about 75 per cent of all 
students have a moderate or strong sense of belonging at school and 80 per cent attend 
classes regularly; they are not consistently absent. Moreover, the findings suggest that there are a 
number of students who are engaged in school, even though they are from low socio-economic families or 
have relatively weak literacy skills. Thus, these students are resilient in that they maintain a positive stance 
towards schooling despite the personal challenges they face. 

Any claims about the prevalence of youths who are disaffected depend on how the construct is defined 
and measured and on where the critical cut-point is set for determining who is at risk and who is not 
(Willms, 2002a). It could be argued that the definitions used in this report lead to rather liberal estimates 
of the prevalence of disaffected youths. However, they may also be too conservative. On affective measures 
such as sense of belonging, students tend to report a rather positive view. Those who indicated that they 
agreed, either moderately or strongly, with statements asserting they felt “lonely” or “like an outsider” 
may be even more disaffected from school than their responses indicate. Similarly, the cut-point for the 
measure of absenteeism is set at a rather conservative level; students were not considered to have a low 
level of participation unless they had missed a considerable amount of school in the previous two weeks. 
Unfortunately, there is no universal definition of truancy and therefore international comparative data on 
truancy rates are not available. When tracking truancy within a country, it would be helpful to document 
how many students are truant on particular days during the school year and how many youths have epi- 
sodes of prolonged truancy. 

There are seven other robust findings concerning student engagement that have important implications 
for social policy. They are robust in that they are statistically significant, large in substantive terms and 
consistent across the majority of countries that participated in the study. Some of these findings challenge 
conventional wisdom about student engagement. 
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(a) The prevalence of disaffected students varies significantly among schools. In nearly every 
country, there is a wide range among schools in the prevalence of students considered to have a low 
sense of belonging and low participation. This variation is only partially attributable to the family 
background of students attending each school. This suggests that there are aspects of school policy and 
practice that contribute to the success of schools in reducing student disaffection. 

(b) At school level, the correlation between sense of belonging and participation is about 
0.37 and both measures of student engagement are moderately correlated (on average, 
about 0.50) with student performance in reading, mathematics and science. These results 
provide evidence that achieving strong student engagement does not have to be at the expense of the 
development of literacy skills. In fact, schools that have strong student engagement tend to have strong 
literacy performance. 

(c) Students’ sense of belonging and participation are not strongly related to each other 
at the individual level and both of these dimensions of student engagement are only 
weakly related with literacy performance. Many studies treat student engagement as a uni- 
dimensional construct and assume that the process of disengaging from school results in poor academic 
performance. Others maintain that those who cannot succeed academically are prone to disengage- 
ment because schooling offers no intrinsic or extrinsic rewards. The results of this study suggest that 
there may be other processes, both in and out of school, that lead to student disaffection. 

(d) On average, across all OECD countries, only about one-quarter (26%) of all students 
have high levels of engagement and strong literacy skills. Another group, also comprising 
about one-quarter of all students (27%), are highly engaged at school and have average levels of literacy 
skills. The remaining students can be classified in three groups: one that has a low sense of belonging, 
but has fairly high literacy performance (about 20% of all students), another that is regularly absent 
from school and has moderately low levels of literacy performance (about 10% of all students) and a 
third group that has very low literacy performance, but average levels of engagement (about 1 7% of 
all students) . These results challenge the popular notion that student disaffection and poor literacy per- 
formance go hand-in-hand. Instead, it suggests that there are many disaffected students who have fairly 
high levels of literacy skills and vice versa. The five-cluster pattern provided a practical classification in 
25 of the 42 countries in this study. A three- or four-cluster categorisation provided a better descrip- 
tion of students in other countries; in these countries also there was no group that had low scores on 
the engagement measures and the measures of academic achievement. 

(e) There are three dominant risk factors for student disaffection: living in a family' of low 
socio-economic status (i.e., in the lowest national quartile for the country ), living in 
a single-parent family and being foreign-born. Living in a high socio-economic status family 
(i.e., the top national quartile) was a relatively weak protective factor for low sense of belonging, but 
was not significantly associated with low participation. Females were equally as likely as males to have 
a low sense of belonging, but were less likely to be regularly absent from school. 

(f) Students who attend schools where there is a concentration of students from low socio- 
economic families are more likely to be disaffected from school. The findings suggest that 
some, but not all, of the negative contextual effect is attributable to factors such as the disciplinary cli- 
mate of the school and student-teacher relations. Some of the contextual effect may also be attributable 
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to peer effects — it might be expected, for example, that students would be more prone to disaffection if 
their friends were also disengaged. 

(g) Schools have higher levels of student engagement when there is a strong disciplinary 
climate, good student-teacher relations and high expectations for student success. 

Levels of school resources do not appear to play a strong role in affecting student engagement. The 
effects associated with school size were inconsistent and relatively weak. Students were less prone to 
low participation in schools with smaller student-staff teaching ratios, but this factor was unrelated to 
students’ sense of belonging. Rather, student engagement has more to do with the culture of the school 
and teachers and principals can play a strong role is creating a positive culture. 

Implications for public policy 

The social policy of a country, a schooling system, or a community, is concerned mainly with achieving 
particular outcomes for its citizens, especially for those who are vulnerable. From this perspective, the 
findings of this study raise three important questions: 

• Does engagement matter?; 

• Can schools effect meaningful change? and if so; 

• What policies and practices lead to higher levels of student engagement? 

These questions are discussed in the light of the findings of this research. 

Does engagement matter? It may be that student disengagement follows a similar course as anti social 
behaviour, with an early onset group setting the base-level prevalence during the elementary school years 
and a large late onset group causing a dramatic increase in the prevalence during the secondary school years 
(Moffit, 1993). Deviant behaviour and attitudes tend to increase sharply during early adolescence, peak at 
about age 16 or 17 and then decrease slowly (Raudenbush and Chan, 1993;Tonry et al., 1991). Research- 
ers have found a similar pattern for criminal behaviour across many societies, leading Gottfredson and 
Hirschi (1990) to refer to the pattern as one of the brute facts of criminology. Thus, when assessing student 
engagement at age 1 5 in PISA, the peak of the curve may be approached when students are least engaged 
in school. It might be argued, therefore, that feeling disaffected from school or withdrawing from school 
life are a normal part of adolescent development and that most youths will re-engage in a significant way 
after the period of compulsory schooling. 

One counter -argument is that student disaffection is a precursor to other activities that render youths 
vulnerable to more serious problems. Other studies have documented a link between disaffection and sub- 
stance abuse (Huizinga et al., 1995), delinquency (Bell et al., 1994) and dropping out of school (Dryfoos, 
1990). Disaffection from school is therefore associated with engaging in activities that can have long-term 
harmful consequences. If family and school interventions can dampen the rise of disaffection during ado- 
lescence, it is likely to reduce the prevalence of more serious problems. The findings of this study indicate 
that the prevalence of disaffection varies among countries and among schools within countries, which sug- 
gests that high levels of disaffection do not have to be the norm . 

Another important counter -argument is that the last two or three years of secondary school are a criti- 
cal transition period for most youths. Their success at school during this period strongly influences their 
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access to post-secondary education and for those who do not plan to pursue further schooling, it affects 
their entry to the labour market. This study suggests that in most countries there is a significant cluster of 
students who are disaffected from school, or have low levels of participation, even though they have rela- 
tively strong literacy skills. It cannot he discerned from these data whether these students are less likely to 
pursue post-secondary schooling, but it is reasonable to infer that students’ attitudes to school during this 
critical period affect their decisions. For those who enter the labour market, engagement is also a critical 
outcome. Many employers are less concerned with their workers’ academic credentials than they are with 
whether they can work well with others, contribute new ideas and align themselves with the goals of the 
organisation (Conference Board of Canada, 1999; OECD, 200 It). It might be expected that students who 
are aligned with the goals of schooling will subsequently be engaged in their work environments, leading 
to success in the labour market. In the case of extreme levels of student disaffection from school, several 
studies have suggested that disaffection and truancy in particular are associated with marital problems, 
violence, adult criminality and incarceration (Baker et al., 2001). 

Finally, student engagement is associated with several positive social outcomes. Research from a number 
of fields has provided compelling evidence that feeling included and being engaged in social pursuits are 
essential to people’s health and wellbeing (Keating and Hertzman, 1999; Putnam, 2000). Increasing 
student engagement will thus contribute to the quality of life of youths, which is important in its own 
right, without consideration of its effect on their development of literacy skills or long-term health and 
economic wellbeing. 

Can schools efTect meaningful change? The findings of this study offer compelling evidence that 
many schools do hav e high levels of engagement and that this is not achieved at the expense of developing 
literacy skills. On average, schools with high levels of engagement tended to have high levels of literacy 
skills. 

The results also suggest that the prevalence of disaffected students varies considerably among schools 
within each country. In some countries, the prevalence of disaffection is relatively low and may not be 
a national priority. However, in these countries disaffection from school may be a concern in particular 
schools or particular regions of the country. This finding calls for more detailed studies of engagement and 
disaffection within countries. In many countries, it would be relatively inexpensive to include measures of 
student engagement in their national monitoring programmes. This would enable policy analysts to con- 
duct detailed within- country analyses. 

The findings also suggest that creating smaller schools, or reducing class sizes, is not necessarily the best 
solution. Rather, the findings indicate that certain school processes — the disciplinary climate of the school, 
student-teacher relations and achievement press — play a bigger role in affecting student engagement. 

What policies and practices lead to higher levels of student engagement? Reforms aimed at 
helping particular students who are disaffected from school usually require some mechanism for identify- 
ing youths who could benefit from particular types of programmes or services. However, targeting pro- 
grammes and services to those who need them the most is a difficult issue (Offord et al . , 1999). One of the 
most surprising findings of this research is that in every country there is a sizeable proportion of students 
who have high levels of literacy skills yet lack a sense of belonging at school or have low levels of participa- 
tion. Educators cannot thus presume that youths with average or above-average literacy skills necessarily 
feel they belong at school, or value schooling outcomes. It seems that many students lack a sense of belong- 
ing hut suffer in silence, while others display their disaffection by withdrawing from school activities or 
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skipping classes. The results indicate that in many countries students who are foreign-born and those from 
single-parent families and low socio-economic families are more prone to being disaffected from school. 
These relationships are sufficiently strong to call for educators to be sensitive to these issues, hut are not 
strong enough to warrant targeting programmes to groups on this basis. 

Taken together, these findings suggest that targeting programmes to particular students is likely to be dif- 
ficult in most schools. Schools may require two or three different kinds of programme to meet the needs of 
disaffected students and these need to be designed with considerable attention to access and selection. For 
example, some students may be disengaged from school because they lack knowledge about post-second- 
ary education or do not have a good understanding of the links between particular school programmes and 
employment opportunities. Others may lack a sense of belonging mainly because of their perceived ability 
to make friends and be accepted among their peers. Some students may withdraw from school activities 
because they seldom or never experience the intrinsic rewards of schooling that are felt by those who 
experience success in academic pursuits. 

Many educators would argue that addressing the problem of student disaffection requires whole-school 
reform. They argue that schools need to be restructured to create smaller learning communities, emphasise 
a core academic programme, eliminate tracking or streaming, achieve greater parental involvement, give 
teachers and students greater autonomy and evaluate students and schools in more authentic ways (Crevola 
and Hill, 1998; Fullan, 2001; Lee and Smith, 1993). Other reformers argue that increasing student 
engagement requires curricular reform. For example, Wehlage et al. (1989) contend that “the dominant 
learning process pursued in schools is too narrow in that it is highly abstract, verbal, sedentary, individual- 
istic, competitive and controlled by others as opposed to concrete, problem-oriented, active, kinaesthetic, 
cooperative and autonomous” . They also maintain that educators are overly obsessed with covering the 
subject matter, as opposed to pursuing a small number of topics in depth and focusing on acquiring greater 
competency in the use of skills and knowledge. 

PISA does not include information that is detailed enough to assess the likely effects of efforts to restruc- 
ture schools. However, this research does provide some guidance as to what elements are key to improv- 
ing student engagement. The results indicate that students are more engaged in schools where there is a 
strong disciplinary climate, positive student-teacher relations, and high expectations for student success. 
The results also show that students who attend schools where there is a concentration of students from 
low socio-economic families are more likely to be disaffected from school. Efforts to reduce streaming or 
tracking and promote social inclusion are thus likely to increase levels of student engagement. 

Concluding remarks. Accommodating diversity is a central challenge for educators and policy-makers. 
This report suggests that in every country there is a substantial number of youths who are disengaged from 
school at the critical period when they are completing their final years of compulsory schooling. Meet- 
ing the needs of these students is critical if they are to use the knowledge and skills they have acquired to 
participate as active citizens in mainstream society. PISA can provide only a broad understanding of the 
nature of student engagement, how it varies among countries and schools and how it is related to family 
and school factors. The findings in this study point to the need for more detailed, focussed research within 
countries aimed at understanding these relationships, assessing current policy and practice and evaluating 
interventions. This study also found that within every country there are many students who are engaged 
in schooling and many schools that are highly effective. Their success gives a reason to be optimistic and 
provides an image of what is possible. 
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PISA measure of student engagement 

PISA 2000 student questionnaire includes items to measure two components of student engagement at 
school — sense of belonging and participation. Student sense of belonging is measured using the question: 

School is a place where: 

a I feel like an outsider (or left out of things) 
b I make friends easily 
c I feel like I belong 
d I feel awkward and out of place 
e Other students seem to like me 
f I feel lonely 

g I do not want to go to school 
h I often feel bored. 

Students were asked to indicate how they felt about each item using a four-point scale: “strongly disagree”, 
“disagree”, “agree” and “strongly agree”. The positive items (b, c and e) were simply scored 1,2,3 and 4, 
and the negative items (a, d, f, g and h) were reverse-scored 4, 3, 2 and 1 , thereby yielding items with high 
scores indicating more positive attitudes. 

A factor analysis of the responses to this question revealed that the responses comprise two factors, one 
that is based primarily on the first six items and describes whether students feel accepted and included by 
their classmates, and another that is based primarily on the last two items and describes whether students 
like school and find it interesting. The analysis also revealed that the six belonging items contributed almost 
equally to the first factor. 1 Therefore, the measure of sense of belonging used in this report is based on a 
Rasch scaling of the first six items, and standardised to have a mean of 500 and a standard deviation of 100 
across all participating OECD countries, similar to the measures of reading, mathematics and science per- 
formance used in PISA. Data from items g and h were not used in the analysis. Although these two items 
are arguably related to the construct, sense of belonging, they did not correlate strongly with the other 
items comprising the overall scale. It appears there is a separate aspect of engagement related to youth 
being bored with school, but these two items were not sufficient to construct a separate scale. 

The measure of student participation was based on a question that asked students how frequently they had 
been absent from school: 

How many times in the previous two weeks did you . . . 

• miss school? 

• skip classes? 

• arrive late for school? 

The possible responses were on a four-point scale, “none”, “ 1 or 2”, “3 or 4” and “5 or more” . 
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Deriving a scale based on this item was difficult, because a high response on one item can lead to a low 
response on another. For example, a student who misses school a number of times is less likely to skip 
classes or be late for school. Similarly, some students who arrive late for school may decide to skip the first 
class altogether, rather than arriving late for class. 

Also, in terms of the amount of instruction a student misses, the three aspects of low participation do not 
deserve an equal weighting. Therefore, the participation scale for this report assigned scores as follows: for 
missing school, the four responses (“none”, “ 1 or 2”, “3 or 4” and “5 or more”) were coded 6, 4, 2, and 0; 
for skipping classes they were coded 4.5, 3.0, 1 . 5, and 0; and for arriving late to school they were coded 
3,2,1 and O.The participation scale was then the sum of the three item scores, which was also scaled to 
have a mean of 500 and a standard deviation of 100 for all participating OECD countries. 

The reliability of a measure refers to the consistency of measurement for repeated measurements of the 
same phenomenon. Often researchers are interested in how well a test instrument can differentiate among 
individuals in a group in their true scores for some abstract construct, based on a set of observed scores. 
But observed scores include measurement error — a portion of the score that is not directly related to 
the construct being assessed (Ward et al., 1996). The estimate of reliability depends on the magnitude of 
measurement error relative to the variation in true scores among the group being assessed. Reliability 
coefficients range from 0.0 to 1.0, and when they are high (approaching 1 .0) it suggests that individuals in 
a group can consistently be differentiated. 

Reliability of measurement can also pertain to how well differentiations among observations describing 
groups, such as the mean score in reading performance in a school, or the prevalence of disaffected youth 
in a country, can be made. In such cases, researchers are concerned with whether the measurement pro- 
cess can consistently differentiate among groups in their observed scores. In this study, most of the analyses 
are concerned with the reliability of estimates of sense of belonging and participation at the school level, 
or at the country level, hi these cases, reliability depends not only on the magnitude of the measurement 
error at the individual level, but also on how accurately school or country means (or other statistics such as 
regression coefficients) are estimated. The size of the within-school samples affects reliability, as does the 
amount of variation among groups in the statistic being assessed (Rowan et al., 1991). 

The measures of sense of belonging and participation are highly reliable at the country level: the reliability 
coefficients are 0.99 for both sense of belonging and participation. Thus, the measurement process helps 
to differentiate among countries in their mean scores with a high degree of consistency. The measures are 
less reliable at the school level — 0.38 for sense of belonging, and 0.60 for participation — for the full set 
of OECD schools. This varies somewhat among countries, ranging from 0. 10 to 0.76 for sense of belong- 
ing, and from 0.38 to 0.87 for participation. The estimates for each country are shown in Annex Table 
A. 1 . These lower estimates suggest that schools cannot be distinguished as reliably as countries, hi most 
countries, PISA data do not furnish estimates that are accurate enough to report findings for a particular 
school. The relatively lower estimates of reliability also mean that the analyses that attempt to estimate the 
relationships of these variables with measures of schooling processes will lack statistical power. Thus, the 
analyses in the last section, which estimate the effects associated with a range of school-level variables, are 
limited to the full sample of OECD students and schools. 
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Table A. 1 

Reliability of the sense of belonging and participation indices 


Sense of belonging 

Participation 

Australia 

0. 56 

0.45 

Austria 

0.28 

0.66 

Belgium 

0.49 

0.82 

Canada 

0.51 

0.47 

Czech Republic 

0.45 

0.62 

Denmark 

0.54 

0.58 

Finland 

0.21 

0.61 

France 

0.50 

0.70 

Germany 

0.28 

0.65 

Greece 

0.45 

0.64 

Hungary 

0.55 

0.71 

Iceland 

0.10 

0.49 

Ireland 

0.51 

0.55 

Italy 

0.59 

0.81 

Japan 

0.58 

0.87 

Korea 

0.55 

0.62 

Luxembourg 

0.76 

0.78 

Mexico 

0.59 

0.56 

New Zealand 

0.22 

0.51 

Norway 

0.41 

0.46 

Poland 

0.52 

0.71 

Portugal 

0.55 

0.49 

Spain 

0.47 

0.72 

Sweden 

0.26 

0.41 

Switzerland 

0.45 

0.62 

United Kingdom 

0.27 

0.58 

United States 

0.46 

0.70 

Albania 

0.49 

0.68 

Argentina 

0.62 

0.72 

Brazil 

0.54 

0.45 

Bulgaria 

0.54 

0.67 

Chile 

0.58 

0.75 

Hong Kong- China 

0.56 

0.64 

Indonesia 

0.42 

0.47 

Israel 

0.48 

0.56 

Latvia 

0.52 

0.6 

Liechtenstein 

0.46 

0.08 

FYR Macedonia 

0.66 

0.61 

Peru 

0.65 

0.61 

Russian Federation 

0.45 

0.59 

Thailand 

0.55 

0.65 

Netherlands 1 

0.57 

0.54 


1 . Response rate is too low to ensure comparability 


Notes 

1 . Principal components factor analysis with varimax rotation was used for the weighted sample of all students in participating 
OECD countries, except The Netherlands. As in the first international report, The Netherlands was excluded because its 
response rate to the survey did not meet the standards set for PISA. The factor weights for the first principal component of 
the rotated matrix were: 0.579, 0.777, 0.629, 0.529, 0.730, 0.595, 0.096, and 0.011. The weights for the second factor 
were: 0.378, -0.017, 0.248, 0.494, -0.115, 0.425, 0.803, and 0.791. 
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Table 2.1 Table 2.2 

Country mean and standard deviation for the index Country mean and standard deviation for the index 

of sense of belonging of participation 




Standard 




Standard 



Mean index 

error 

Standard deviation 


Mean index 

error 

Standard deviation 

Australia 

+95 

(2.0) 

97 

gj Australia 

502 

(2.1) 

89 

Austria 

526 

(2.3) 

109 

H Austria 

513 

(2.2) 

85 

Belgium 

479 

(1.3) 

90 

O Belgium 

518 

(1.7) 

9+ 

Canada 

512 

(1.1) 

110 

0 Canada 

+81 

(1.1) 

10+ 

Czech Republic 

+71 

(1.6) 

78 

O Czech Republic 

+93 

(2.2) 

99 

Denmark 

513 

(2.2) 

10+ 

Denmark 

+61 

(2.4) 

12+ 

Finland 

502 

(1.4) 

96 

Finland 

+88 

(2.1) 

103 

France 

+86 

(1.6) 

9+ 

France 

512 

(2.1) 

93 

Germany 

518 

(1.8) 

107 

Germany 

523 

(1.9) 

85 

Greece 

+98 

(2.0) 

95 

Greece 

+75 

(2.7) 

112 

Hungary 

51+ 

(1.6) 

97 

Hungary 

509 

(1.9) 

96 

Iceland 

51+ 

(1.8) 

109 

Iceland 

+8+ 

(1.8) 

110 

Ireland 

508 

(1.7) 

101 

Ireland 

503 

(2.1) 

89 

Italy 

500 

(1.6) 

92 

Italy 

+8+ 

(2.6) 

98 

Japan 

+65 

(1.9) 

89 

Japan 

555 

(1.9) 

57 

Korea 

+61 

(1.6) 

81 

Korea 

5+6 

(IS) 

71 

Luxembourg 

505 

(1.8) 

110 

Luxembourg 

515 

(14) 

96 

Mexico 

509 

(2.2) 

98 

Mexico 

+98 

(2.1) 

89 

New Zealand 

+98 

(1.9) 

98 

New Zealand 

+79 

(2.1) 

110 

Norway 

512 

(2.2) 

10+ 

Norway 

503 

(2.0) 

102 

Poland 

+61 

(1.9) 

85 

Poland 

+77 

(3.7) 

119 

Portugal 

501 

(1.9) 

88 

Portugal 

50+ 

(18) 

91 

Spain 

+99 

(1.6) 

91 

Spain 

+72 

(2.5) 

118 

Sweden 

527 

(1.8) 

103 

Sweden 

+89 

(1.5) 

99 

Switzerland 

520 

(2.0) 

105 

Switzerland 

515 

(1.9) 

90 

United Kingdom 

513 

(1-4) 

101 

United Kingdom 

509 

(IS) 

86 

United States 

+94 

(3.1) 

111 

United States 

+9+ 

(3.9) 

100 

OECD average 

500 

(0.4) 

100 

OECD average 

500 

(0.4) 

100 

Albania 

+59 

(1.6) 

80 

Albania 

515 

(2.1) 

89 

Argentina 

518 

(3.7) 

107 

Argentina 

+71 

(6.2) 

12+ 

Brazil 

522 

(2.4) 

102 

Brazil 

+66 

(2.9) 

109 

Bulgaria 

+81 

(1.9) 

85 

Bulgaria 

4+1 

(3.4) 

133 

Chile 

519 

(2.3) 

110 

Chile 

+7+ 

(2.9) 

111 

Hong Kong- China 

+58 

(1.3) 

73 

Hong Kong- China 

557 

(1.2) 

51 

Indonesia 

+79 

(1.7) 

72 

Indonesia 

522 

(1.7) 

79 

Israel 

5+4 

(2.9) 

115 

Israel 

+28 

(5.3) 

129 

Latvia 

+6+ 

(2.1) 

79 

Cg Latvia 

+83 

(2.7) 

103 

Liechtenstein 

521 

(5.5) 

113 

2 Liechtenstein 

537 

(4-1) 

79 

FYR Macedonia 

513 

(1.7) 

98 

g FYR Macedonia 

+99 

(1.6) 

109 

Peru 

480 

(2.5) 

99 

q Peru 

+73 

(2.5) 

113 

Russian Federation 

+75 

(1.6) 

85 

Russian Federation 

+80 

(2.5) 

11+ 

Thailand 

+69 

(IS) 

77 

O Thailand 

+89 

(2.1) 

97 

Netherlands' 

+99 

(2.8) 

8+ 

Netherlands' 

+99 

(2.8) 

92 


1 . Response rate is too low to ensure comparability. 1 . Response rate is too low to ensure comparability. 
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Table 2.3 

Percentages of students who have a low sense of belonging 
and low participation 



Low sense 

of belonging 

Low 

participation 


% 

Standard error 

% 

Standard error 

Australia 

20.7 

(0.8) 

18.3 

(0.8) 

Austria 

20.3 

(0.7) 

15.3 

(0.8) 

Belgium 

31.6 

(0.6) 

14.1 

(0.6) 

Canada 

20.5 

(0.4) 

26.0 

(0.5) 

Czech Republic 

29.8 

(0.7) 

20.7 

(0.8) 

Denmark 

20.9 

(0.7) 

32.9 

(0.9) 

Finland 

21.3 

(0.7) 

22.9 

(0.9) 

France 

30.2 

(0.7) 

15.3 

(0.7) 

Germany 

22.6 

(0.6) 

12.9 

(0.7) 

Greece 

22.7 

(0.9) 

28.8 

(1.0) 

Hungary 

18.8 

(0.6) 

17.7 

(0.7) 

Iceland 

22.4 

(0.7) 

26.0 

(0.8) 

Ireland 

19.4 

(0.7) 

17.8 

(0.7) 

Italy 

22.9 

(0.8) 

21.7 

(0.9) 

Japan 

37.6 

(1.0) 

4.2 

(0.6) 

Korea 

41.4 

(1.1) 

8.4 

(0.6) 

Luxembourg 

28.3 

(0.8) 

13.4 

(0.5) 

Mexico 

22.0 

(0.9) 

21.4 

(0-8) 

New Zealand 

21.1 

(0.8) 

26.9 

(0.9) 

Norway 

21.1 

(0.8) 

17.9 

(0.8) 

Poland 

41.2 

(12) 

29.2 

(1.3) 

Portugal 

20.7 

(0.9) 

20.1 

(0.7) 

Spain 

24.0 

(0.7) 

34.0 

(1.0) 

Sweden 

17.7 

(0.5) 

23.8 

(0.6) 

Switzerland 

20.8 

(0.7) 

15.7 

(0.7) 

United Kingdom 

17.4 

(0.6) 

15.0 

(0.6) 

United States 

25.0 

(1.0) 

20.2 

(1.1) 

OECD average 

24.5 

(0.1S) 

20.0 

(0.1S) ■ 

Albania 

39.7 

(0.9) 

15.0 

(0.8) 

Argentina 

21.9 

(1.7) 

28.4 

(2.6) 

Brazil 

17.1 

(0.7) 

31.8 

(1.2) 

Bulgaria 

29.0 

<M> 

40.5 

(1.1) 

Chile 

23.6 

(0.9) 

28.4 

(1.2) 

Hong Kong- China 

33.4 

(0.8) 

3.3 

(0.3) 

Indonesia 

23.8 

(1.1) 

14.5 

(0.6) 

Israel 

18.5 

(0.9) 

45.4 

(19) 

Latvia 

36.0 

(1.1) 

28.0 

(1.3) 

Liechtenstein 

23.9 

(2.1) 

9.1 

(1.7) 

FYR Macedonia 

22.9 

(0.7) 

21.2 

(0.6) 

Peru 

36.9 

(1.2) 

31.2 

(1.0) 

Russian Federation 

33.4 

(1.0) 

30.0 

(0.9) 

Thailand 

32.7 

(0.9) 

25.4 

(0.9) 

Netherlands 1 

20.1 

(1.2) 

20.0 

(1.2) 

1 . Response rate is too low to ensure 

comparability. 
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Table 3.1 

Student-level correlations among five outcome measures 


Sense of belonging - 
Participation 

Sense of belonging - 
Reading literacy 

Sense of belonging - 
Mathematical literacy 

Sense of belonging - 
Scientific literacy 

Participation - 
Reading literacy 

Participation - 
Mathematical literacy 

Participation - 
Scientific literacy 

Reading literacy - 
Mathematical literacy 

Reading literacy - 
Scientific literacy 

Mathematical literacy - 
Scientific literacy 

Australia 

0.03 

0.02 

0.01 

0.02 

0.12 

0.16 

0.12 

0.79 

0.88 

0.79 

Austria 

0.0+ 

0.07 

0.08 

0.05 

0.09 

0.08 

0.10 

0.67 

0.80 

0.62 

Belgium 

0.07 

0.05 

0.02 

0.02 

0.16 

0.18 

0.16 

0.71 

0.75 

0.69 

Canada 

0.02 

0.0+ 

0.02 

0.01 

0.20 

0.18 

0.19 

0.77 

0.86 

0.7+ 

Czech Republic 

0.07 

0.11 

0.11 

0.08 

0.18 

0.18 

0.15 

0.67 

0.77 

0.67 

Denmark 

0.09 

0.0+ 

0.06 

0.01 

0.16 

0.20 

0.16 

0.81 

0.85 

0.72 

Finland 

0.00 

-0.0+ 

-0.03 

-0.03 

0.18 

0.18 

0.18 

0.70 

0.81 

0.71 

France 

0.06 

0.03 

0.05 

0.01 

0.18 

0.13 

0.18 

0.57 

0.69 

0.61 

Germany 

0.03 

0.00 

-0.03 

0.01 

0.10 

0.10 

0.10 

0.65 

0.77 

0.6+ 

Greece 

0.09 

0.09 

0.03 

0.10 

0.13 

0.11 

0.12 

0.57 

0.65 

0.65 

Hungary 

0.03 

0.09 

0.03 

0.03 

0.06 

0.06 

0.0+ 

0.63 

0.68 

0.4+ 

Iceland 

0.0+ 

0.02 

0.01 

-0.01 

0.2+ 

0.22 

0.22 

0.77 

0.80 

0.68 

Ireland 

0.05 

0.00 

0.00 

0.02 

0.1+ 

0.09 

0.16 

0.81 

0.89 

0.81 

Italy 

0.03 

-0.01 

-0.01 

0.00 

0.1+ 

0.12 

0.12 

0.66 

0.75 

0.59 

Japan 

0.10 

-0.02 

0.01 

-0.05 

0.12 

0.10 

0.13 

0.60 

0.73 

0.57 

Korea 

0.08 

0.05 

0.02 

0.06 

0.11 

0.15 

0.13 

0.63 

0.73 

0.66 

Luxembourg 

0.10 

0.16 

0.11 

0.1+ 

0.13 

0.13 

0.13 

0.77 

0.83 

0.75 

Mexico 

0.10 

0.1+ 

0.10 

0.05 

0.05 

0.02 

0.03 

0.62 

0.70 

0.6+ 

New Zealand 

0.03 

0.05 

0.03 

0.02 

0.19 

0.15 

0.23 

0.83 

0.90 

0.80 

Norway 

0.08 

0.06 

0.02 

0.05 

0.20 

0.18 

0.19 

0.76 

0.85 

0.76 

Poland 

0.01 

0.07 

0.0+ 

0.07 

0.11 

0.0+ 

0.11 

0.61 

0.69 

0.65 

Portugal 

0.0+ 

0.20 

0.13 

0.1+ 

0.07 

0.07 

0.09 

0.78 

0.81 

0.67 

Spain 

0.0+ 

0.05 

0.01 

0.03 

0.18 

0.19 

0.17 

0.7+ 

0.81 

0.66 

Sweden 

0.08 

-0.0+ 

-0.01 

-0.05 

0.13 

0.15 

0.15 

0.81 

0.86 

0.75 

Switzerland 

0.06 

0.08 

0.08 

0.07 

0.13 

0.09 

0.09 

0.70 

0.79 

0.63 

United Kingdom 

0.09 

0.06 

0.06 

0.03 

0.21 

0.22 

0.2+ 

0.81 

0.87 

0.83 

United States 

0.08 

0.13 

0.15 

0.12 

0.19 

0.22 

0.20 

0.81 

0.87 

0.76 

OECD average 

0.07 

0.06 

0.04 

0.04 

0.14 

0.13 

0.14 

0.1 7 

0.79 

0.68 

Albania 

0.05 

0.10 

0.12 

0.11 

0.1+ 

0.11 

0.09 

0.62 

0.68 

0.51 

Argentina 

0.07 

0.13 

0.06 

0.13 

0.16 

0.12 

0.15 

0.61 

0.66 

0.+2 

Brazil 

0.07 

0.10 

0.02 

0.0+ 

0.11 

0.07 

0.09 

0.6+ 

0.61 

0.+7 

Bulgaria 

0.08 

0.15 

0.15 

0.08 

0.15 

0.12 

0.09 

0.62 

0.66 

0.55 

Chile 

0.05 

0.10 

0.09 

0.0+ 

0.1+ 

0.10 

0.11 

0.67 

0.67 

o.+o 

Hong Kong- China 

0.08 

0.06 

0.05 

0.01 

0.12 

0.10 

0.11 

0.67 

0.76 

0.59 

Indonesia 

0.06 

0.15 

0.06 

0.1+ 

0.05 

0.06 

0.06 

0.+8 

0.+5 

0.+9 

Israel 

0.05 

0.06 

0.00 

0.00 

0.09 

0.08 

0.10 

0.58 

0.67 

0.66 

Latvia 

0.07 

0.08 

0.08 

0.08 

0.1+ 

0.11 

0.1+ 

0.63 

0.72 

0.73 

Liechtenstein 

0.1+ 

0.16 

0.09 

0.08 

0.22 

0.1+ 

0.25 

0.60 

0.79 

0.+9 

FYR Macedonia 

0.05 

0.13 

0.11 

0.03 

0.15 

0.09 

0.10 

0.68 

0.72 

0.62 

Peru 

0.08 

0.20 

0.07 

0.09 

0.10 

0.08 

0.03 

0.+9 

0.50 

0.38 

Russian Federation 

0.07 

0.08 

0.09 

0.07 

0.13 

0.07 

0.08 

0.6+ 

0.72 

0.51 

Thailand 

0.13 

0.19 

0.15 

0.17 

0.27 

0.20 

0.21 

0.68 

0.76 

0.65 

Netherlands 1 

0.01 

0.01 

-0.02 

-0.03 

0.16 

0.23 

0.13 

0.69 

0.79 

0.67 


1. Response rate is too low to ensure comparability. 
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Table 3.2 

School-level correlations among five outcome measures 


I 

H 

s 

O 

0 

1 

o 


Sense of belonging - 
Participation 

Sense of belonging - 
Reading literacy 

Sense of belonging - 
Mathematical literacy 

Sense of belonging - 
Scientific literacy 

Participation - 
Reading literacy 

Participation - 
Mathematical literacy 

Participation - 
Scientific literacy 

Reading literacy - 
Mathematical literacy 

Reading literacy - 
Scientific literacy 

Mathematical literacy - 
Scientific literacy 

Australia 

0.47 

0.48 

0.51 

0.48 

0.32 

0.35 

0.37 

0.98 

0.99 

1.00 

Austria 

0.09 

0.44 

0.34 

0.42 

-0.02 

0.04 

0.01 

0.95 

0.99 

0.98 

Belgium 

0.25 

0.37 

0.31 

0.35 

0.91 

0.90 

0.89 

0.98 

0.99 

0.99 

Canada 

0.05 

0.23 

0.19 

0.22 

0.18 

0.27 

0.21 

0.95 

0.99 

0.97 

Czech Republic 

0.56 

0.51 

0.52 

0.53 

0.67 

0.73 

0.71 

0.95 

0.98 

0.99 

Denmark 

0.08 

0.48 

0.47 

0.41 

0.26 

0.29 

0.28 

1.00 

1.00 

0.99 

Finland 

0.23 

0.43 

0.37 

0.37 

0.01 

0.21 

0.03 

0.96 

0.98 

0.96 

France 

0.26 

0.41 

0.38 

0.41 

0.76 

0.77 

0.77 

0.99 

1.00 

1.00 

Germany 

0.46 

0.79 

0.76 

0.73 

0.46 

0.53 

0.53 

0.99 

0.98 

1.00 

Greece 

0.51 

0.58 

0.55 

0.58 

0.65 

0.63 

0.63 

0.99 

1.00 

1.00 

Hungary 

0.84 

0.97 

0.95 

0.97 

0.88 

0.85 

0.87 

0.98 

0.99 

0.99 

Iceland 

0.54 

0.27 

0.23 

0.27 

0.01 

0.03 

0.08 

1.00 

1.00 

0.99 

Ireland 

0.09 

-0.17 

-0.23 

-0.19 

0.55 

0.44 

0.55 

0.89 

0.96 

0.98 

Italy 

0.21 

0.01 

-0.05 

-0.02 

0.91 

0.88 

0.89 

0.95 

0.99 

0.98 

Japan 

0.56 

0.60 

0.64 

0.59 

0.69 

0.67 

0.67 

0.98 

0.99 

1.00 

Korea 

0.57 

0.63 

0.77 

0.73 

0.69 

0.63 

0.68 

0.94 

0.97 

0.99 

Luxembourg 

0.65 

0.82 

0.75 

0.81 

0.73 

0.64 

0.70 

0.98 

1.00 

0.99 

Mexico 

0.25 

0.74 

0.71 

0.72 

0.09 

0.08 

0.08 

1.00 

1.00 

0.99 

New Zealand 

0.05 

-0.07 

-0.01 

-0.05 

0.72 

0.73 

0.79 

0.94 

0.96 

0.99 

Norway 

0.12 

0.55 

0.45 

0.52 

0.47 

0.64 

0.48 

0.96 

0.96 

0.97 

Poland 

0.62 

0.81 

0.80 

0.82 

0.67 

0.69 

0.69 

0.98 

0.99 

0.99 

Portugal 

0.13 

0.92 

0.94 

0.94 

0.19 

0.22 

0.22 

0.99 

1.00 

1.00 

Spain 

0.08 

0.32 

0.08 

0.23 

0.39 

0.40 

0.31 

0.93 

0.97 

0.96 

Sweden 

0.08 

0.39 

0.48 

0.37 

0.10 

0.22 

0.07 

0.98 

0.98 

0.94 

Switzerland 

0.55 

0.49 

0.43 

0.48 

0.15 

0.15 

0.18 

0.98 

0.99 

0.99 

United Kingdom 

0.22 

0.15 

0.20 

0.15 

0.69 

0.67 

0.70 

0.99 

0.99 

0.99 

United States 

0.40 

0.37 

0.25 

0.34 

0.69 

0.65 

0.70 

0.98 

1.00 

0.99 

OECD average 

0.37 

0.51 

0.48 

0.50 

0.48 

0.50 

0.49 

0.97 

0.99 

0.M 

Albania 

0.32 

0.15 

0.24 

0.18 

0.59 

0.56 

0.56 

0.96 

0.97 

0.96 

Argentina 

0.62 

0.92 

0.88 

0.91 

0.80 

0.80 

0.79 

0.99 

0.99 

1.00 

Brazil 

0.65 

0.53 

0.51 

0.50 

0.76 

0.70 

0.73 

0.98 

0.99 

1.00 

Bulgaria 

0.86 

0.93 

0.90 

0.91 

0.73 

0.70 

0.71 

0.97 

0.98 

1.00 

Chile 

0.81 

0.83 

0.82 

0.82 

0.68 

0.66 

0.64 

0.99 

0.99 

1.00 

Hong Kong- China 

0.10 

0.70 

0.68 

0.70 

0.59 

0.58 

0.58 

0.99 

0.99 

1.00 

Indonesia 

0.16 

0.59 

0.58 

0.59 

0.33 

0.37 

0.41 

0.98 

0.98 

1.00 

Israel 

0.16 

0.39 

0.35 

0.23 

-0.24 

-0.25 

-0.16 

0.99 

0.95 

0.94 

Latvia 

0.09 

0.72 

0.77 

0.67 

0.22 

0.26 

0.19 

0.98 

0.99 

0.98 

Liechtenstein 

0.25 

0.25 

0.17 

0.27 

0.58 

0.54 

0.47 

0.97 

0.99 

0.96 

FYR Macedonia 

0.42 

0.83 

0.77 

0.84 

0.48 

0.44 

0.38 

0.98 

0.97 

0.97 

Peru 

0.38 

0.86 

0.85 

0.81 

0.49 

0.53 

0.53 

0.99 

0.98 

1.00 

Russian Federation 

0.40 

0.52 

0.48 

0.46 

0.41 

0.42 

0.39 

0.96 

0.99 

0.98 

Thailand 

0.79 

0.67 

0.68 

0.64 

0.52 

0.51 

0.52 

0.97 

0.98 

0.99 

Netherlands 1 

0.59 

0.68 

0.64 

0.68 

0.72 

0.73 

0.70 

1.00 

1.00 

1.00 


1 . Response rate is too low to ensure comparability. 
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■ ANNEX B 


Tabled 

Categories of students based on a cluster analysis of their sense of belonging, 
participation and reading and mathematical literacy performance 



Student category 

Percentage of 
students 

Cluster mean 

Sense of belonging 

Participation 

Reading literacy 

Mathematical 

literacy 

Australia 

Top students 

2fi_S 

470 

534 

640 

628 


Engaged students 

12_1 

650 

517 

544 

543 


Students feeling isolated 

ioa 

449 

534 

501 

518 


Absentee students 

1QJ3 

462 

323 

531 

526 


Non-academic students 

li_8 

482 

497 

378 

408 


All Clusters 

100.0 

495 

502 

528 

533 

Austria 

Top students 

76 6 

599 

531 

592 

599 


Engaged students 

24-5 

617 

533 

460 

465 


Students feeling isolated 

74.8 

425 

540 

552 

549 


Absentee students 

SL6 

493 

328 

515 

515 


Non -academic students 

14-5 

448 

524 

373 

390 


All Clusters 

100.0 

526 

513 

507 

514 

Belgium 

Well-rounded students 

719 

597 

539 

540 

540 


Academic students who feel isolated 

44.? 

442 

548 

585 

593 


Non-academic, disaffected students 

34.9 

447 

529 

389 

410 


Non-academic, absentee students 

£L2 

449 

272 

392 

410 


All Clusters 

100.0 

479 

518 

507 

515 

Canada 

Top students 

15-2 

612 

512 

614 

599 


Engaged students 

16 4 

640 

498 

484 

485 


Students feeling isolated 

32-1 

441 

512 

601 

590 


Absentee students 

ms 

493 

262 

487 

495 


Non-academic students 

73 7 

444 

503 

443 

455 


All Clusters 

100.0 

512 

481 

534 

533 

Czech Republic 

Top students 

2fL5 

473 

526 

611 

620 


Engaged students 

ms 

592 

525 

531 

533 


Students feeling isolated 

32-1 

434 

518 

510 

509 


Absentee students 

12_Q 

451 

301 

436 

428 


Non -academic students 

30.0 

454 

523 

407 

400 


All Clusters 

100.0 

471 

493 

492 

493 

Denmark 

Top students 

79 6 

469 

524 

578 

583 


Engaged students 

US 

650 

511 

514 

530 


Students feeling isolated 

26-1 

464 

511 

408 

442 


Absentee students 

13_D 

498 

308 

533 

542 


Non-academic students 

2_6 

498 

211 

369 

399 


All Clusters 

100.0 

513 

461 

497 

514 

Finland 

Well-rounded students 

219 

626 

511 

548 

535 


Academic, disaffected students 

39 4 

466 

521 

619 

597 


Non-academic, disaffected students 

22-1 

460 

514 

467 

471 


Non-academic, absentee students 

116 

497 

267 

495 

494 


All Clusters 

100.0 

502 

488 

546 

536 

France 

Well-rounded students 

74.8 

594 

536 

529 

543 


Academic, disaffected students 

32.2 

446 

540 

579 

586 


Non-academic, disaffected students 

79 1 

444 

528 

429 

442 


Non -academic, absentee students 

8_9 

475 

285 

419 

446 


All Clusters 

100.0 

486 

512 

505 

517 

Germany 

Top students 

23-3 

570 

546 

605 

603 


Engaged students 

24-5 

621 

541 

468 

469 


Students feeling isolated 

22-2 

423 

538 

526 

530 


Absentee students 

3-2 

484 

276 

407 

415 


Non-academic students 

19 0 

463 

529 

361 

374 


All Clusters 

100.0 

518 

523 

484 

485 

Greece 

Top students 

23-3 

467 

496 

578 

566 


Engaged students 

16-6 

641 

527 

520 

480 


Students feeling isolated 

30-1 

454 

512 

464 

436 


Absentee students 

11-2 

480 

242 

421 

384 


Non -academic students 

16-5 

497 

487 

354 

306 


All Clusters 

100.0 

498 

475 

474 

447 
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Table 3.3 (continued) 

Categories of students based on a cluster analysis of their sense of belonging, 
participation and reading and mathematical literacy performance 






Cluster 

mean 




Perc entage of 




Mathematical 


Student category 

students 

Sense of belonging 

Participation 

Reading literacy 

literacy 

Hungary 

Well-rounded students 

4- 5 9 

541 

538 

556 

565 


Non-academic, absentee students 

11.9 

502 

293 

405 

418 


Non-academic students 

42.-4: 

481 

531 

409 

417 


All Clusters 

100.0 

514 

509 

480 

483 

Iceland 

Well-rounded students 

2-3J5 

647 

515 

517 

523 


Academic, disaffected students who 
feel isolated 

-33_1 

460 

520 

584 

585 


Non-academic students who feel 
isolated 

28,6 

474 

524 

434 

455 


Non-academic, absentee students 

14 9 

494 

273 

452 

474 


All Clusters 

100.0 

514 

483 

507 

514 

Ireland 

Top students 

IS S 

556 

533 

630 

592 


Engaged students 

2IL0 

633 

521 

504 

480 


Students feeling isolated 


432 

523 

569 

537 


Absentee students 


486 

298 

482 

483 


Non-academic students 

70 4 

467 

518 

420 

407 


All Clusters 

100.0 

508 

503 

527 

503 

Italy 

Well-rounded students 

48 .7 

492 

520 

557 

527 


Non-academic students who feel 
isolated 

12.0 

479 

278 

416 

397 


Non- academic students 

19.1 

512 

502 

424 

397 


All Clusters 

100.0 

500 

484 

487 

457 

Japan 

Top students 

74 9 

452 

566 

618 

650 


Engaged students 

21A 

580 

569 

540 

572 


Students feeling isolated 

17.1 

412 

563 

524 

552 


Absentee students 

2_9 

410 

292 

406 

446 


Non-academic students 

17 6 

442 

547 

402 

439 


All Clusters 

100.0 

465 

555 

522 

557 

Korea 

Well-rounded students 

2S_S 

563 

565 

558 

588 


Academic students who feel isolated 

16 6 

417 

561 

564 

598 


Non-academic students who feel 
isolated 

3£L2 

441 

561 

458 

470 


Non-academic, absentee students 

2J1 

425 

341 

465 

479 


All Clusters 

100.0 

461 

546 

525 

547 

Luxembourg 

Well-rounded students 

50 7 

547 

542 

514 

510 


Non-academic students who feel 
isolated 

ZiS 

464 

263 

396 

407 


Non- academic students 

4±_& 

457 

535 

371 

393 


All Clusters 

100.0 

505 

515 

441 

446 

Mexico 

Top students 

1S.1 

608 

512 

516 

474 


Engaged students 

71.1 

593 

533 

388 

358 


Students feeling isolated 

77 7 

444 

513 

484 

447 


Absentee students 

11.5 

470 

342 

386 

360 


Non-academic students 

74. 8 

434 

526 

336 

309 


All Clusters 

100.0 

509 

498 

422 

387 

Netherlands 1 

Well-rounded students 

74 9 

592 

526 

589 

615 


Academic students who feel isolated 

33_1 

449 

530 

593 

620 


Non-academic students who feel 
isolated 

78 1 

487 

520 

453 

483 


Non-academic, absentee students 

11.6 

483 

313 

481 

497 


All Clusters 

100.0 

499 

500 

532 

563 

New Zealand 

Top students 

17,0 

476 

507 

633 

636 


Engaged students 

14 6 

668 

488 

538 

546 


Students who feel isolated 

7S 6 

456 

523 

516 

522 


Absentee students 

10.7 

478 

240 

459 

487 


Non -academic students 

14 7 

471 

478 

367 

399 


All Clusters 

100.0 

498 

479 

529 

537 

Norway 

Well-rounded students 

2421 

636 

529 

519 

505 


Academic students who feel isolated 

40 7 

468 

529 

581 

567 


Non-academic students who feel 
isolated 

77 7 

472 

519 

400 

412 


Non-academic, absentee students 

3-2 

499 

236 

425 

443 


All Clusters 

100.0 

512 

503 

505 

499 


1_ Response rate is too low to ensure comparability. 
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Table 3-3 (continued) 

Categories of students based on a cluster analysis of their sense of belonging, 
participation and reading and mathematical literacy performance 





Cluster mean 



Percentage of 




Mathematical 


Student category 

students 

Sense of belonging 

Participation 

Reading literacy 

literacy 

Poland 

Top students 

iff- i 

442 

538 

572 

564 


Engaged students 

1£lA 

583 

527 

504 

489 


Students who feel isolated 

14.8 

448 

373 

520 

510 


Absentee students 

8-2 

436 

207 

384 

388 


Non -academic students 

?9 S 

419 

513 

395 

386 


All Clusters 

100.0 

461 

477 

479 

470 

Portugal 

Well-rounded students 

20.2 

627 

517 

495 

474 


Academic students who feel isolated 

36 S 

483 

517 

551 

531 


Non- academic students who feel 
isolated 

34 0 

454 

536 

383 

379 


Non-academic, absentee students 

2-2 

467 

303 

395 

395 


All Clusters 

100.0 

501 

504 

470 

454 

Spain 

Well-rounded students 

24.2 

601 

514 

514 

491 


Academic students who feel isolated 

33.6 

457 

518 

565 

554 


Non-academic students who feel 
isolated 

TLA 

464 

493 

414 

396 


Non-academic, absentee students 

14.6 

482 

242 

456 

435 


All Clusters 

100.0 

499 

472 

493 

476 

Sweden 

Top students 

23.1 

538 

524 

619 

615 


Engaged students 

25.3 

627 

525 

498 

495 


Students who feel isolated 

24 0 

430 

518 

528 

522 


Absentee students 

10.7 

519 

296 

506 

504 


Non-academic students 

16 9 

491 

481 

384 

381 


All Clusters 

100.0 

527 

489 

516 

510 

Switzerland 

Top students 

23 5 

581 

527 

611 

636 


Engaged students 

232Z 

618 

542 

471 

498 


Students who feel isolated 

TLA 

433 

536 

528 

563 


Absentee students 

1A 

493 

289 

453 

494 


Non- academic students 

IS 0 

457 

535 

362 

402 


All Clusters 

100.0 

520 

515 

494 

529 

United Kingdom 

Top students 

22-1 

495 

539 

645 

638 


Engaged students 

18.1 

669 

532 

534 

540 


Students who feel isolated 

32.6 

464 

531 

527 

531 


Absentee students 

8-8 

493 

311 

466 

479 


Non- academic students 

TLA 

474 

515 

392 

415 


All Clusters 

100.0 

513 

509 

523 

529 

United States 

Top students 

30.4 

457 

535 

606 

587 


Engaged students 

IQ S 

670 

524 

526 

515 


Students who feel isolated 

30.4 

439 

534 

451 

446 


Absentee students 

8-S 

460 

334 

518 

497 


Non -academic students 

10 Q 

447 

368 

336 

348 


All Clusters 

100.0 

494 

494 

504 

493 

Albania 

Top students 

24.3 

458 

544 

463 

508 


Engaged students 

12-S 

615 

531 

350 

389 


Disaffected students 

22-0 

433 

541 

355 

383 


Truant students 

8_8 

429 

311 

331 

372 


Non- academic students 

22.5 

431 

528 

238 

258 


All Clusters 

100.0 

459 

515 

349 

380 

Argentina 

Top students 

18.7 

585 

513 

549 

540 


Engaged students 

21.1 

635 

512 

407 

374 


Disaffected students 

2S2Z 

455 

504 

463 

442 


Truant students 

14.5 

492 

240 

333 

311 


Non -academic students 

70 0 

445 

512 

302 

277 


All Clusters 

100.0 

518 

471 

418 

388 

Brazil 

Top students 

25.1 

485 

515 

475 

438 


Engaged students 

20-2 

655 

524 

405 

336 


Disaffected students 

3122 

466 

510 

339 

272 


Truant students 

112 

577 

321 

426 

370 


Non- academic students 

11-8 

468 

290 

306 

252 


All Clusters 

100.0 

521 

466 

396 

334 
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Table 3.3 (continued) 

Categories of students based on a cluster analysis of their sense of belonging, 
participation and reading and mathematical literacy performance 



Student category 

Percentage of 
students 

Cluster mean 

Sense of belonging 

Participation 

Reading literacy 

Mathematical 

literacy 

Bulgaria 

Well-rounded students 

4 2-2 

503 

493 

519 

526 


Non- academic, disaffected students 

19.6 

460 

230 

374 

376 


Non-academic students 

38. ? 

464 

500 

372 

368 


All Clusters 

100.0 

481 

441 

430 

430 

Chile 

Top students 

79 8 

662 

509 

487 

461 


Engaged students 

70 a 

463 

505 

481 

457 


Disaffected students 

17 7 

424 

500 

329 

297 


Truant students 

12JZ 

488 

251 

336 

321 


Non- academic students 

12-2 

596 

514 

377 

348 


All Clusters 

100.0 

519 

473 

410 

383 

Hong Kong- China 

Top students 

314 

442 

566 

605 

644 


Engaged students 

13.6 

593 

566 

561 

594 


Disaffected students 

45-7 

432 

563 

515 

538 


Truant students 

4-0 

434 

336 

457 

490 


Non-academic students 

14.8 

436 

552 

398 

413 


All Clusters 

100.0 

458 

557 

525 

560 

Indonesia 

Well-rounded students 

44-8 

499 

547 

422 

433 


Non-academic, disaffected students 

13. 6 

469 

368 

346 

345 


Non-academic students 

41.6 

459 

546 

317 

302 


All Clusters 

100.0 

479 

522 

371 

367 

Israel 

Well-rounded students 

43-1 

568 

471 

530 

533 


Non-academic, disaffected students 

18 5 

505 

218 

462 

442 


Non- academic students 

38 4 

525 

482 

369 

334 


All Clusters 

100.0 

545 

429 

452 

433 

Latvia 

Top students 

79-1 

463 

507 

561 

572 


Engaged students 

16-0 

575 

513 

457 

459 


Disaffected students 

24-6 

418 

528 

434 

438 


Truant students 

41—4 

450 

283 

414 

444 


Non- academic students 

44-0 

437 

462 

309 

324 


All Clusters 

100.0 

464 

483 

458 

463 

Liechtenstein 

Top students 

44-4 

504 

547 

603 

641 


Engaged students 

37.0 

636 

558 

505 

523 


Disaffected students 

7 6 6 

419 

557 

494 

519 


Truant students 

4-2 

486 

284 

431 

471 


Non- academic students 

44-2 

467 

525 

348 

373 


All Clusters 

100.0 

521 

537 

483 

514 

FYR Macedonia 

Top students 

20,1 

541 

525 

491 

507 


Engaged students 

70 9 

626 

527 

373 

378 


Disaffected students 

7d.fi 

442 

529 

401 

409 


Truant students 

40-4 

477 

259 

327 

347 


Non- academic students 

719 

442 

527 

271 

278 


All Clusters 

100.0 

503 

499 

372 

379 

Peru 

Top students 

20.4 

504 

509 

453 

439 


Engaged students 

16.3 

627 

525 

334 

285 


Disaffected students 

28.6 

430 

516 

326 

305 


Truant students 

44-4 

467 

286 

309 

283 


Non-academic students 

18.9 

421 

486 

221 

177 


All Clusters 

100.0 

480 

473 

327 

292 

Russian Federation 

Well-rounded students 

44-2 

487 

528 

528 

560 


Non- academic, disaffected students 

46-3 

463 

277 

434 

467 


Non -academic students 

32.4 

465 

514 

390 

395 


All Clusters 

100.0 

475 

480 

462 

478 

Thailand 

Well-rounded students 

43_3 

503 

536 

501 

506 


Non- academic, disaffected students 

18 6 

440 

328 

382 

395 


Non- academic students 

38-4 

447 

517 

390 

385 


All Clusters 

100.0 

469 

489 

431 

432 
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Table 11 

Student characteristics and family background factors associated with low sense of belonging (odds ratios) 1 


Female students 

High socio-economic 
background' 

Low socio-economic 
background' 

Foreign-born student* 

Single-parent family 

Australia 

0.96 

(LSI 

121 

114 

120 

Austria 

1.01 

1 04 

125 

128 

118 

Belgium 

0.87 

0.90 

1.38 

141 

117 

Canada 

0.85 

0.79 

1.39 

159 

124 

Czech Republic 

0.89 

0.83 

1.48 

— 

107 

Denmark 

1.27 

0.86 

1.24 

1 06 

118 

Finland 

1.36 

1 00 

-L-U 

— 

1.77 

France 

1 03 

0.78 

121 

0.97 

1 15 

Germany 

1.01 

0.81 

132 

149 

102 

Greece 

034 

-L02 

1 -33 

139 

124 

Hungary 

0.91 

0.77 

135 

— 

1 is 

Iceland 

1 01 

0.85 

12 x 2 

159 

1 77 

Ireland 

0.97 

TOO 

UK 

7.79 

1.15 

Italy 

0.90 

0.96 

1.73 

— 

1.11 

Japan 

0.90 

0.86 

132 

— 

120 

Korea 

1 . 4-0 

0.85 

147 

— 

121 

Luxembourg 

0.95 

0-73 

164 

130 

149 

Mexico 

0.86 

0274 

146 

187 

122 

New Zealand 

1.04 

0.87 

136 

131 

1.13 

Norway 

m 

0.78 

160 

135 

1 06 

Poland 

0.92 

0.80 

176 

— 

1 00 

Portugal 

1.05 

0.70 

184 

115 

111 

Spain 

0 - 84 - 

0.91 

125 

— 

118 

Sweden 

1.56 

1.05 

141 

121 

1.05 

Switzerland 

112 

0.91 

134 

132 

125 

United Kingdom 

0.85 

0.95 

131 

120 

1 11 

United States 

0 75 

0 71 

1 58 

161 

1 33 

OECD average 

0.98 ,c 

0 . 86 c 

1 . 3 S C 

137 * 

1 AJ 

Albania 

1.02 

0.91 

122 

— 

1.03 

Argentina 

0.89 

0.53 

137 

— 

126 

Brazil 

0-77 

0.92 

170 

— 

1 - 1-3 

Bulgaria 

0.75 

0278 

165 

— 

115 

Chile 

0.86 

0-71 

132 

— 

118 

Hong Kong- China 

0.89 

0.76 

137 

no 

129 

Indonesia 

0.92 

0.68 

1.30 

— 

0.98 

Israel 

0.61 

0.89 

126 

170 

128 

Latvia 

0.88 

0229 

1,61 

1 , 7 ? 

1,18 

Liechtenstein 

0.72 

0.75 

106 

189 

1.27 

FYR Macedonia 

0.78 

0.66 

142 

137 

0.89 

Peru 

0.88 

0 61 

136 

— 

0.90 

Russian Federation 

0.90 

0.85 

150 

114 

105 

Thailand 

0-65 

0.77 

134 

— 

1 10 

Netherlands 4 

1,08 

0.98 

127 

126 

0.84 


-1_ Odds ratios in bold text are statistically significant (p < 0.05). For the OECD average, odds ratios with a superscript s vary significantly among schools, 
and those with a superscript c vary significantly among countries. 

2. Students in top and bottom national quartiles of die socio-economic status measered by International Socio-economic Index of Occupational Status (ISEI) 
of parents. 

1 Odds ratios are not reported for countries in which less than 3% of the students are foreign-born. 

1 Response rate is too low to ensure comparability. 
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Table 4*2 


Student characteristics and family background factors associated with low participation (odds ratios) 1 



Female students 

High socio-economic 
background* 

Low socio-economic 
background* 

Foreign- 

-born student* 

Single-parent family 

Australia 

1*10 


1*08 

1*27 


1*02 


1-55 


Austria 

1.06 


1*18 

0.97 


1*21 


1-63 


Belgium 

0.93 


0*78 

1-63 


1*98 


1-23 


Canada 

0.98 


0.96 

1*24 


0.98 


1*22 


Czech Republic 

0.96 


0 81 

1*27 


— 


1*18 


Denmark 

1.10 


0.91 

1*15 


1*32 


1*47 


Finland 

1.01 


0*81 

1*10 


— 


1*55 


France 

0.88 


0.91 

1*48 


1*51 


1*50 


Germany 

0.98 


1*22 

1*24 


0.99 


1*51 


Greece 

(L67 


0.95 

1*07 


1*19 


1*11 


Hungary 

0.85 


0-81 

1*57 


— 


1*10 


Iceland 

0.98 


0*74 

1*52 


1*19 


0.95 


Ireland 

0.77 


0.85 

1*55 


1*33 


1*23 


Italy 

0*71 


0.93 

1*12 


— 


1*13 


Japan 

0.70 


1*12 

1.70 


— 


1*41 


Korea 

0,84 


0.98 

1*01 


— 


1,50 


Luxembourg 

0.90 


0.87 

1*16 


1.46 


1*30 


Mexico 

0,80 


1 09 

0,8Q 


1,79 


1.7? 


New Zealand 

1*17 


0.88 

1*52 


0.94 


1*30 


Norway 

0.96 


0.85 

1*21 


2.00 


1.30 


Poland 

0.75 


1*21 

1.00 


— 


1.29 


Portugal 

0.86 


0.99 

0.91 


1*13 


1*44 


Spain 

L13 


0.85 

1.31 


— 


1*43 


Sweden 

1.13 


1*13 

1.50 


1 00 


1*51 


Switzerland 

0.98 


1*32 

0.94 


1.67 


1*44 


United Kingdom 

0.89 


0.78 

1*74 


1*31 


1.56 


United States 

106 


0.86 

1*61 


0.97 


1.30 


OECD average 

0.93 c 


0.94 c 

1*261 



■1 

Hr i r 4o 


Albania 

0.40 


0.86 

JL12 


— 


1.00 


Argentina 

0 77 


0.73 

1*14 


— 


1*24 


Brazil 

0.86 


0*78 

1236 


— 


1*38 


Bulgaria 

0.83 


1 06 

1*24 


— 


1*10 


Chile 

0.93 


0.93 

1.20 


— 


1.20 


Hong Kong- China 

0.87 


1.78 

0.99 


1*31 


1.67 


Indonesia 

0.61 


1 07 

1*12 


— 


0.99 


Israel 

1 11 


0.92 

0-76 


0-76 


1*12 


Latvia 

0.75 


.LOT 

1*17 


1*17 


111 


Liechtenstein 

2.58 


0.88 

0 8? 


2*78 


0*55 


FYR Macedonia 

0.68 


1.19 

1*05 


0.93 


1*26 


Peru 

0.85 


0.93 

1*22 


— 


1*23 


Russian Federation 

0*75 


0.99 

1*18 


0.98 


1*11 


Thailand 

0.50 


0.85 

1*02 


— 


1*07 


Netherlands 4 

0.99 


0.97 

1.38 


JJ2 


1*34 


1* Odds ratios in bold text are statistically significant (p < 0*05)* For the OECD average, odds ratios widi a superscript s 
and those with a superscript c vary significandy among countries. 

2- Students in top and bottom national quartiles of die socio-economic status measered by International Socio-economic 
of parents. 

3* Odds ratios are not reported for countries in which less than 3% of the students are foreign-born. 

4* Response rate is too low to ensure comparability. 

vary significandy among schools, 
Index of Occupational Status (ISEI) 
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Tabled 


Country mean on the indices of disciplinary climate, student-teacher relations and achievement press 


Disciplinary climate 

Student-teacher relations 

Achievement press 

Australia 

4.2 

6-2 

5.2 

Austria 

6D 

1-1 

1-2 

Belgium 

4,6 

-5-1 

1-2 

Canada 

4_L 

6-1 

6-4 

Czech Republic 

5-6 

4-1 

1-2 

Denmark 

4_£ 

6-4 

5-2 

Finland 

4_L 

-5-2 

5-2 

France 

4-1 

427 

4-6 

Germany 

5_4 

1-2 

4-4 

Greece 

2-5 

4-1 

1-5 

Hungary 

5-1 

5-2 

6-4 

Iceland 

5-6 

4-1 

2-5 

Ireland 

5-6 

5-1 

6-1 

Italy 

1-2 

4-4 

6-1 

Japan 

211 

2-6 

Q.6 

Korea 

5.2 

5.1 

1-4 

Luxembourg 

5-1 

2-2 

1-1 

Mexico 

6-1 

5-5 

4-1 

New Zealand 

4-6 

6-6 

6-1 

Norway 

1-2 

4-1 

5_1 

Poland 

6-1 

2.1 

6D 

Portugal 

4_2 

6-1 

42Z 

Spain 

4_L 

4-2 

4.5 

Sweden 

1.1 

6-2 

5_5 

Switzerland 

6-2 

6-1 

4-2 

United Kingdom 

5-1 

6.6 

2-1 

United States 

5-4 

6-1 

6-4 

OECD average 

5-2 

511 

5D 

Albania 

1-2 

4-1 

5_5 

Argentina 

1-4 

4-2 

2-2 

Brazil 

1-2 

2D 

4-2 

Bulgaria 

4-1 

4D 

3-2 

Chile 

2-2 

4-4 

5D 

Hong Kong- China 

4-1 

5.4 

2D 

Indonesia 

6-6 

5-5 

1-2 

Israel 

121 

5-1 

1-1 

Latvia 

2-1 

4-1 

6_L 

Liechtenstein 

2-2 

5-1 

4-5 

FYR Macedonia 

1-1 

4-5 

2-2 

Peru 

4-2 

5-6 

44 

Russian Federation 

2-2 

5-1 

6-5 

Thailand 

2-1 

2-5 

4-2 

Netherlands 1 

i-1 

6-4 

2-2 

1_ Response rate is too low to ensure 

comparability. 
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THE DEVELOPMENTOFTHE 
PISA THEMATIC REPORT 
- A COLLABORATIVE EFFORT 


■ ANNEX C 


Introduction 

PISA is a collaborative effort, bringing together scientific expertise from the participating countries, 
steered jointly by their governments on the basis of shared, policy-driven interests. 

A Board of Participating Countries on which each country is represented determines, in the context of 
OECD objectives, the policy priorities for PISA and oversees adherence to these priorities during the 
implementation of the programme. This includes the setting of priorities for the development of indica- 
tors, for the establishment of the assessment instruments and for the reporting of the results. 

Experts from participating countries also serve on working groups that are charged with linking policy 
objectives with the best internationally available technical expertise. By participating in these expert 
groups, countries ensure that: the instruments are internationally valid and lake into account the cultural 
and educational contexts in OECD Member countries; the assessment materials have strong measurement 
properties; and the instruments place an emphasis on authenticity and educational validity. 

Through National Project Managers, participating countries implement PISA at the national level subject 
to the agreed administration procedures. National Project Managers play a vital role in ensuring that the 
implementation of the survey is of high quality, and verify and evaluate the survey results, analyses, reports 
and publications. 

The design and implementation of the surveys, within the framework established by the Board of Partici- 
pating Countries, is the responsibility of the PISA consortium, referred to as the PISA Consortium, led 
by the Australian Council for Educational Research (ACER). Other partners in this consortium include 
the Netherlands National Institute for Educational Measurement (Citogroep), The National Institute for 
Educational Research in Japan (NIER), the Educational Testing Service in the United States (ETS), and 
WESTAT in the United States. 

The OECD Secretariat has overall managerial responsibility for the programme, monitors its implementa- 
tion on a day-to-day basis, acts as the secretariat for the Board of Participating Countries, builds consensus 
among countries and serves as the interlocutor between the Board of Participating Countries and the 
international consortium charged with the implementation of the activities. The OECD Secretariat also 
produces the indicators and analyses and prepares the international reports and publications in co-opera- 
tion with the PISA consortium and in close consultation with Member countries both at the policy level 
(Board of Participating Countries) and at the level of implementation (National Project Managers). 

The following lists the members of the various PISA bodies and the individual experts and consultants who 
have contributed to PISA during the first cycle. 
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Members of the PISA Board of Participating 
Countries (PISA 2000 and PISA Plus) 

Chair: Eugene Owen 

Argentina: Lilia Toranzos 
Albania: Vladimir Pasko and Nikoleta Mika 
Australia: Wendy Whitham 
Austria: Friedrich Plank 

Belgium: Dominique Barthelemy, Christiane Blondin. 
Dominique Lafontaine, Liselotte van de Perre 
Brazil: Maria Helena Guimaraes de Castro 
Bulgaria: Alexander Petkov Lakiurski 
Canada: Satya Brink, Patrick Bussiere, Dianne Pennock 
Chile: Leonor Cariola 

Czech Republic: Jan Koucky, Jana Strakova 
Denmark: Birgitte Bovin 
Finland: Ritva Jakku-Sihvonen 
France: Gerard Bonnet 

Germany: Jochen Schweitzer, Helga Hinke, Gudrun Stoltenberg 

Greece: Vassilis Koulaidis 

Hong Kong-China : Esther Sui Chu Ho 

Hungary: Peter Vari 

Iceland: Einar Gudmundsson 

Indonesia: Ramon Mohandas, Bahrul Hayat 

Ireland : Gerry Shiel 

Israel: Bracha Kramarski, Zemira Mevarech 

Italy: Chiara Croce, Elisabetta Midena, Benedetto Vertecchi 

Japan: RyoWatanabe 

Korea: Kooghyang Ro 

Latvia: Andris Kangro 

Luxembourg: Jean- Paul Reeff 

FYR Macedonia: Vladimir Mostrov 

Mexico: Fernando Cordova Calderon 

Netherlands: Arnold Spee 

New Zealand: Lynne Whitney 

Norway: Alette Schreiner 

Peru: Nancy Tor re jon, Ana Pasco Font de Tirado, Giuliana 
Espinosa 

Poland: Kazimierz Korab 
Portugal: Gloria Ramalho 
Romania: Adrian Stoica, Roxana Mihail 
Russian Federation: Galina Kovalyova 
Spain: Guillermo Gil 

Sweden: Anders Auer, Birgitta Fredander, Anita Wester 
Switzerland: Heinz Gilomen 
Thailand: Sunee Klainin 

United Kingdom: Lorna Bertrand, Brian Semple 
United States: Mariann Lemke 


PISA National Project Managers 
(PISA 2000 and PISA Plus) 

Argentina: Lilia Toranzos 
Albania: Vladimir Pasko and Nikoleta Mika 
Australia: Jan Lokan 
Austria: Gunter Haider 

Belgium: Dominique Lafontaine, Luc van de Poele 
Brazil: Tereza Cristina Cotta, Maria Lucia Guardia, 

Maria Ines Pestana 
Bulgaria: Alexander Petkov Lakiurski 
Canada: Marc Lachance, Dianne Pennock 
Chile: Leonor Cariola 
Czech Republic: Jana Strakova 
Denmark: Vita Bering Pruzan 
Finland: Jouni Valijarvi 
France: Jean- Pierre Jeantheau 
Germany: Juergen Baumert, Petra Stanat 
Greece: Katerina Kassotakis 
Hong Kong-China: Esther Sui Chu Ho 
Hungary: PeterVari 

Iceland: Julius Bjornsson, Ragna Benedikta Gardarsdottir 

Indonesia: Ramon Mohandas, Bahrul Hayat 

Ireland: Judith Cosgrove 

Israel: Bracha Kramarski, Zemira Mevarech 

Italy: Emma Nardi 

Japan: RyoWatanabe 

Korea: Kooghyang Ro 

Latvia: Andris Kangro 

Luxembourg: Iris Blanke, Jean-Paul Reeff 

FYR Macedonia: Vladimir Mostrov 

Mexico: Fernando Cordova Calderon 

Netherlands: JohanWijnstra 

New Zealand: Steve May 

Norway: Svein Lie 

Peru: Nancy Tor rej on, Ana Pasco Font de Tirado, 

Giuliana Espinosa 
Poland: Michal Federowicz 
Portugal: Gloria Ramalho 
Romania: Adrian Stoica, Roxana Mihail 
Russian Federation: Galina Kovalyova 
Spain: Guillermo Gil 

Sweden: Bengt-Olov Molander, Astrid Pettersson, 

Karin Taube 

Switzerland: Huguette McCluskey 
Thailand: Sunee Klainin 
United Kingdom: Baljit Gill, Graham Thorpe 
United States: Ghedam Bairu, Marilyn Binkley 
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OECD Secretariat 

Andreas Schleicher (overall co-ordination of PISA and 
Member country relations) 

Kooghyang Ro (thematic analyses) 

Claudia Tamassia (project management) 

Hannah Cocks (statistical support) 

Sophie Vayssettes (statistical support) 

Juliet Evans (administrative support) 

UNESCO Institute for Statistics 

Douglas Lynd 
Albert Motivans 
Yanhong Zhang 
Marie-Helene Lussier 

PISA Expert Groups 

Mathematics Functional Expert Group 

Jan de Lange (Chair) (Utrecht University, The Netherlands) 
Raimondo Bolletta (Istituto Nazionale di Valutazione, Italy) 
Sean Close (St Patrick’s College, Ireland) 

Maria Luisa Moreno (IES “ Lope de Vega” , Spain) 

Mogens Niss (IMFUFA, Roskilde University, Denmark) 
Kyungmee Park (Hongik University, Korea) 

Thomas A. Romberg (United States) 

Peter Schuller (Federal Ministry of Education and Cultural 
Affairs, Austria) 

Reading Functional Expert Group 

Irwin Kirsch (Chair) (Educational Testing Service, United States) 
Marilyn Binkley (National Center for Educational Statistics, 
United States) 

Alan Davies (University of Edinburgh, United Kingdom) 

Stan Jones (Statistics Canada, Canada) 

John de Jong (Language Testing Services, The Netherlands) 
Dominique Lafontaine (Universite de liege SartTilman, Belgium) 
Pirjo Linnakyla (University of Jyvaskyla, Finland) 

Martine Remond (Institut National de Recherche Pedagogi- 
que, France) 

Wolfgang Schneider (University ofWiirzburg, Germany) 
RyoWatanabe (National Institute for Educational Research, Japan) 

Science Functional Expert Group 

Wynne Harlen (Chair) (University of Bristol , United Kingdom) 
Peter Fensham (Monash University, Australia) 

Raul Gagliardi (University of Geneva, Switzerland) 

Svein Lie (University of Oslo, Norway) 

Manfred Prenzel (Universitat Kiel, Germany) 

Senta A. Raizen (National Center for Improving Science 
Education (NCISE), Llnited States) 


Donghee Shin (DankooK University, Korea) 

Elizabeth Stage (University of California, United States) 

PISA Technical Advisory Group (PISA) 

Ray Adams (ACER, Australia) 

Pierre Foy (Statistics Canada, Canada) 

Aletta Grisay (Belgium) 

Larry Hedges (The University of Chicago, United States) 
Eugene Johnson (American Institutes for Research, United 
States) 

John de Jong (Language Testing Services, The Netherlands) 
Geoff Masters (ACER, Australia) 

Keith Rust (WESTAT, United States) 

Norman Verhelst (Citogroep, The Netherlands) 

J. Douglas Willms (University of New Brunswick, Canada) 

PISA Consortium (PISA 2000 and PISA Plus) 

Australian Council for Educational Research 

Ray Adams (Project Director of the PISA Consortium) 
Christian Monseur (Project Director of the PISA Consor- 
tium for PISA Plus, Director of the PISA Consortium for data 
processing, data analysis and quality monitoring for PISA 2000) 
Alla Berezner (data processing, data analysis) 

Claus Carstensen (data analysis) 

Lynne Darkin (reading test development) 

Brian Doig (mathematics test development) 

Adrian Harvey-Beavis (quality monitoring, questionnaire 
development) 

Kathryn Hill (reading test development) 

John Lindsey (mathematics test development) 

Jan Lokan (quality monitoring, field procedures development) 
Le Tu Luc (data processing) 

Greg Macaskill (data processing) 

Joy McQueen (reading test development and reporting) 

Gary Marks (questionnaire development) 

Juliette Mendelovits (reading test development and reporting) 
Gayl O’Connor (science test development) 

Alla Routitsky (data processing) 

Wolfram Schulz (data analysis) 

Ross Turner (test analysis and reporting co-ordination) 
Nikolai Volodin (data processing) 

Craig Williams (data processing, data analysis) 

Margaret Wu (Deputy Project Director of the PISA Consortium) 
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Westat 

Nancy Caldwell (Director of the PISA Consortium for field 
operations and quality monitoring) 

Ming Chen (sampling and weighting) 

Fran Cohen (sampling and weighting) 

Susan Fuss (sampling and weighting) 

Brice Hart (sampling and weighting) 

Sharon Hirabayashi (sampling and weighting) 

Sheila Krawchuk (sampling and weighting) 

Dward Moore (field operations and quality monitoring) 

Phu Nguyen (sampling and weighting) 

Monika Peters (field operations and quality monitoring) 
Merl Robinson (field operations and quality monitoring) 
Keith Rust (Director of the PISA Consortium for sampling 
and weighting) 

Leslie Wallace (sampling and weighting) 

Dianne Walsh (field operations and quality monitoring) 
Trevor Williams (questionnaire development) 

Citogroep 

Steven Bakker (science test development) 

Bart Bossers (reading test development) 

Truus Decker (mathematics test development) 

Erna van Hest (reading test development and quality monitoring) 
Kees Lagerwaard (mathematics test development) 

Gerben van Lent (mathematics test development) 

Ico de Roo (science test development) 

Maria vanToor (office support and quality monitoring) 
Norman Verhelst (technical advice, data analysis) 

Educational Testing Service 

Irwin Kirsch (reading test development) 

Other experts (PISA 2000 and PISA Plus) 

Cordula Artelt (questionnaire development) 

Marc Demeuse (quality monitoring) 

Harry Ganzeboom (questionnaire development) 

Aletta Grisay (technical advice, data analysis, translation, 
questionnaire development) 

Donald Hirsch (editorial review) 

Katharina Michaelowa (reporting) 

Jules Peschar (questionnaire development) 

Erich Ramseier (questionnaire development) 

Gundel Schumel (questionnaire development) 
Marie-Andree Somers (data analysis and reporting) 

Peter Sutton (editorial review) 

Rich Tobin (questionnaire development and reporting) 

J. Douglas Willms (questionnaire development, data analysis 
and reporting) 
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